ASPLOS 2022 – Author Index

A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

	Achermann, Reto	ASPLOS '22: "Enzian: An Open, General, ..." Enzian: An Open, General, CPU/FPGA Platform for Systems Software Research David Cock, Abishek Ramdas, Daniel Schwyn, Michael Giardino, Adam Turowski, Zhenhao He, Nora Hossle, Dario Korolija, Melissa Licciardello, Kristina Martsenko, Reto Achermann, Gustavo Alonso, and Timothy Roscoe (ETH Zurich, Switzerland; University of British Columbia, Canada) Hybrid computing platforms, comprising CPU cores and FPGA logic, are increasingly used for accelerating data-intensive workloads in cloud deployments, and are a growing topic of interest in systems research. However, from a research perspective, existing hardware platforms are limited: they are often optimized for concrete, narrow use-cases and, therefore lack the flexibility needed to explore other applications and configurations. We show that a research group can design and build a more general, open, and affordable hardware platform for hybrid systems research. The platform, Enzian, is capable of duplicating the functionality of existing CPU/FPGA systems with comparable performance but in an open, flexible system. It couples a large FPGA with a server-class CPU in an asymmetric cache-coherent NUMA system. Enzian also enables research not possible with existing hybrid platforms, through explicit access to coherence messages, extensive thermal and power instrumentation, and an open, programmable baseboard management processor. Enzian is already being used in multiple projects, is open source (both hardware and software), and available for remote use. We present the design principles of Enzian, the challenges in building it, and evaluate it with a range of existing research use-cases alongside other, more specialized platforms, as well as demonstrating research not possible on existing platforms. @InProceedings{ASPLOS22p434, author = {David Cock and Abishek Ramdas and Daniel Schwyn and Michael Giardino and Adam Turowski and Zhenhao He and Nora Hossle and Dario Korolija and Melissa Licciardello and Kristina Martsenko and Reto Achermann and Gustavo Alonso and Timothy Roscoe}, title = {Enzian: An Open, General, CPU/FPGA Platform for Systems Software Research}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {434--451}, doi = {10.1145/3503222.3507742}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Acun, Bilge	ASPLOS '22: "RecShard: Statistical Feature-Based ..." RecShard: Statistical Feature-Based Memory Optimization for Industry-Scale Neural Recommendation Geet Sethi, Bilge Acun, Niket Agarwal, Christos Kozyrakis, Caroline Trippel, and Carole-Jean Wu (Stanford University, USA; Meta, USA) We propose RecShard, a fine-grained embedding table (EMB) partitioning and placement technique for deep learning recommendation models (DLRMs). RecShard is designed based on two key observations. First, not all EMBs are equal, nor all rows within an EMB are equal in terms of access patterns. EMBs exhibit distinct memory characteristics, providing performance optimization opportunities for intelligent EMB partitioning and placement across a tiered memory hierarchy. Second, in modern DLRMs, EMBs function as hash tables. As a result, EMBs display interesting phenomena, such as the birthday paradox, leaving EMBs severely under-utilized. RecShard determines an optimal EMB sharding strategy for a set of EMBs based on training data distributions and model characteristics, along with the bandwidth characteristics of the underlying tiered memory hierarchy. In doing so, RecShard achieves over 6 times higher EMB training throughput on average for capacity constrained DLRMs. The throughput increase comes from improved EMB load balance by over 12 times and from the reduced access to the slower memory by over 87 times. @InProceedings{ASPLOS22p344, author = {Geet Sethi and Bilge Acun and Niket Agarwal and Christos Kozyrakis and Caroline Trippel and Carole-Jean Wu}, title = {RecShard: Statistical Feature-Based Memory Optimization for Industry-Scale Neural Recommendation}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {344--358}, doi = {10.1145/3503222.3507777}, year = {2022}, } Publisher's Version
	Adams, Andrew	ASPLOS '22: "Vector Instruction Selection ..." Vector Instruction Selection for Digital Signal Processors using Program Synthesis Maaz Bin Safeer Ahmad, Alexander J. Root, Andrew Adams, Shoaib Kamil, and Alvin Cheung (Adobe, USA; Massachusetts Institute of Technology, USA; University of California at Berkeley, USA) Instruction selection, whereby input code represented in an intermediate representation is translated into executable instructions from the target platform, is often the most target-dependent component in optimizing compilers. Current approaches include pattern matching, which is brittle and tedious to design, or search-based methods, which are limited by scalability of the search algorithm. In this paper, we propose a new algorithm that first abstracts the target platform instructions into high-level uber-instructions, with each uber-instruction unifying multiple concrete instructions from the target platform. Program synthesis is used to lift input code sequences into semantically equivalent sequences of uber-instructions and then to lower from uber-instructions to machine code. Using 21 real-world benchmarks, we show that our synthesis-based instruction selection algorithm can generate instruction sequences for a hardware target, with the synthesized code performing up to 2.1x faster as compared to code generated by a professionally-developed optimizing compiler for the same platform. @InProceedings{ASPLOS22p1004, author = {Maaz Bin Safeer Ahmad and Alexander J. Root and Andrew Adams and Shoaib Kamil and Alvin Cheung}, title = {Vector Instruction Selection for Digital Signal Processors using Program Synthesis}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1004--1016}, doi = {10.1145/3503222.3507714}, year = {2022}, } Publisher's Version
	Agarwal, Niket	ASPLOS '22: "RecShard: Statistical Feature-Based ..." RecShard: Statistical Feature-Based Memory Optimization for Industry-Scale Neural Recommendation Geet Sethi, Bilge Acun, Niket Agarwal, Christos Kozyrakis, Caroline Trippel, and Carole-Jean Wu (Stanford University, USA; Meta, USA) We propose RecShard, a fine-grained embedding table (EMB) partitioning and placement technique for deep learning recommendation models (DLRMs). RecShard is designed based on two key observations. First, not all EMBs are equal, nor all rows within an EMB are equal in terms of access patterns. EMBs exhibit distinct memory characteristics, providing performance optimization opportunities for intelligent EMB partitioning and placement across a tiered memory hierarchy. Second, in modern DLRMs, EMBs function as hash tables. As a result, EMBs display interesting phenomena, such as the birthday paradox, leaving EMBs severely under-utilized. RecShard determines an optimal EMB sharding strategy for a set of EMBs based on training data distributions and model characteristics, along with the bandwidth characteristics of the underlying tiered memory hierarchy. In doing so, RecShard achieves over 6 times higher EMB training throughput on average for capacity constrained DLRMs. The throughput increase comes from improved EMB load balance by over 12 times and from the reduced access to the slower memory by over 87 times. @InProceedings{ASPLOS22p344, author = {Geet Sethi and Bilge Acun and Niket Agarwal and Christos Kozyrakis and Caroline Trippel and Carole-Jean Wu}, title = {RecShard: Statistical Feature-Based Memory Optimization for Industry-Scale Neural Recommendation}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {344--358}, doi = {10.1145/3503222.3507777}, year = {2022}, } Publisher's Version ASPLOS '22: "TMO: Transparent Memory Offloading ..." TMO: Transparent Memory Offloading in Datacenters Johannes Weiner, Niket Agarwal, Dan Schatzberg, Leon Yang, Hao Wang, Blaise Sanouillet, Bikash Sharma, Tejun Heo, Mayank Jain, Chunqiang Tang, and Dimitrios Skarlatos (Meta, USA; Carnegie Mellon University, USA) The unrelenting growth of the memory needs of emerging datacenter applications, along with ever increasing cost and volatility of DRAM prices, has led to DRAM being a major infrastructure expense. Alternative technologies, such as NVMe SSDs and upcoming NVM devices, offer higher capacity than DRAM at a fraction of the cost and power. One promising approach is to transparently offload colder memory to cheaper memory technologies via kernel or hypervisor techniques. The key challenge, however, is to develop a datacenter-scale solution that is robust in dealing with diverse workloads and large performance variance of different offload devices such as compressed memory, SSD, and NVM. This paper presents TMO, Meta’s transparent memory offloading solution for heterogeneous datacenter environments. TMO introduces a new Linux kernel mechanism that directly measures in realtime the lost work due to resource shortage across CPU, memory, and I/O. Guided by this information and without any prior application knowledge, TMO automatically adjusts how much memory to offload to heterogeneous devices (e.g., compressed memory or SSD) according to the device’s performance characteristics and the application’s sensitivity to memory-access slowdown. TMO holistically identifies offloading opportunities from not only the application containers but also the sidecar containers that provide infrastructure-level functions. To maximize memory savings, TMO targets both anonymous memory and file cache, and balances the swap-in rate of anonymous memory and the reload rate of file pages that were recently evicted from the file cache. TMO has been running in production for more than a year, and has saved between 20-32% of the total memory across millions of servers in our large datacenter fleet. We have successfully upstreamed TMO into the Linux kernel. @InProceedings{ASPLOS22p609, author = {Johannes Weiner and Niket Agarwal and Dan Schatzberg and Leon Yang and Hao Wang and Blaise Sanouillet and Bikash Sharma and Tejun Heo and Mayank Jain and Chunqiang Tang and Dimitrios Skarlatos}, title = {TMO: Transparent Memory Offloading in Datacenters}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {609--621}, doi = {10.1145/3503222.3507731}, year = {2022}, } Publisher's Version
	Ahmad, Hammad	ASPLOS '22: "CirFix: Automatically Repairing ..." CirFix: Automatically Repairing Defects in Hardware Design Code Hammad Ahmad, Yu Huang, and Westley Weimer (University of Michigan, USA) This paper presents CirFix, a framework for automatically repairing defects in hardware designs implemented in languages like Verilog. We propose a novel fault localization approach based on assignments to wires and registers, and a fitness function tailored to the hardware domain to bridge the gap between software-level automated program repair and hardware descriptions. We also present a benchmark suite of 32 defect scenarios corresponding to a variety of hardware projects. Overall, CirFix produces plausible repairs for 21/32 and correct repairs for 16/32 of the defect scenarios. This repair rate is comparable to that of successful program repair approaches for software, indicating CirFix is effective at bringing over the benefits of automated program repair to the hardware domain for the first time. @InProceedings{ASPLOS22p990, author = {Hammad Ahmad and Yu Huang and Westley Weimer}, title = {CirFix: Automatically Repairing Defects in Hardware Design Code}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {990--1003}, doi = {10.1145/3503222.3507763}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Ahmad, Maaz Bin Safeer	ASPLOS '22: "Vector Instruction Selection ..." Vector Instruction Selection for Digital Signal Processors using Program Synthesis Maaz Bin Safeer Ahmad, Alexander J. Root, Andrew Adams, Shoaib Kamil, and Alvin Cheung (Adobe, USA; Massachusetts Institute of Technology, USA; University of California at Berkeley, USA) Instruction selection, whereby input code represented in an intermediate representation is translated into executable instructions from the target platform, is often the most target-dependent component in optimizing compilers. Current approaches include pattern matching, which is brittle and tedious to design, or search-based methods, which are limited by scalability of the search algorithm. In this paper, we propose a new algorithm that first abstracts the target platform instructions into high-level uber-instructions, with each uber-instruction unifying multiple concrete instructions from the target platform. Program synthesis is used to lift input code sequences into semantically equivalent sequences of uber-instructions and then to lower from uber-instructions to machine code. Using 21 real-world benchmarks, we show that our synthesis-based instruction selection algorithm can generate instruction sequences for a hardware target, with the synthesized code performing up to 2.1x faster as compared to code generated by a professionally-developed optimizing compiler for the same platform. @InProceedings{ASPLOS22p1004, author = {Maaz Bin Safeer Ahmad and Alexander J. Root and Andrew Adams and Shoaib Kamil and Alvin Cheung}, title = {Vector Instruction Selection for Digital Signal Processors using Program Synthesis}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1004--1016}, doi = {10.1145/3503222.3507714}, year = {2022}, } Publisher's Version
	Ahn, Gail-Joon	ASPLOS '22: "ViK: Practical Mitigation ..." ViK: Practical Mitigation of Temporal Memory Safety Violations through Object ID Inspection Haehyun Cho, Jinbum Park, Adam Oest, Tiffany Bao, Ruoyu Wang, Yan Shoshitaishvili, Adam Doupé, and Gail-Joon Ahn (Soongsil University, South Korea; Samsung Research, South Korea; PayPal, USA; Arizona State University, USA; Samsung Research, USA) Temporal memory safety violations, such as use-after-free (UAF) vulnerabilities, are a critical security issue for software written in memory-unsafe languages such as C and C++. In this paper, we introduce ViK, a novel, lightweight, and widely applicable runtime defense that can protect both operating system (OS) kernels and user-space applications against temporal memory safety violations. ViK performs object ID inspection, where it assigns a random identifier to every allocated object and stores the identifier in the unused bits of the corresponding pointer. When the pointer is used, ViK inspects the value of a pointer before dereferencing, ensuring that the pointer still references the original object. To the best of our knowledge, this is the first mitigation against temporal memory safety violations that scales to OS kernels. We evaluated the software prototype of ViK on Android and Linux kernels and observed runtime overhead of around 20%. Also, we evaluated a hardware-assisted prototype of ViK on Android kernel, where the runtime overhead was as low as 2%. @InProceedings{ASPLOS22p271, author = {Haehyun Cho and Jinbum Park and Adam Oest and Tiffany Bao and Ruoyu Wang and Yan Shoshitaishvili and Adam Doupé and Gail-Joon Ahn}, title = {ViK: Practical Mitigation of Temporal Memory Safety Violations through Object ID Inspection}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {271--284}, doi = {10.1145/3503222.3507780}, year = {2022}, } Publisher's Version Artifacts Functional
	Ainsworth, Sam	ASPLOS '22: "MineSweeper: A “Clean Sweep” ..." MineSweeper: A “Clean Sweep” for Drop-In Use-after-Free Prevention Márton Erdős, Sam Ainsworth, and Timothy M. Jones (University of Cambridge, UK; University of Edinburgh, UK) Low-level languages, which require manual memory management from the programmer, remain in wide use for performance-critical applications. Memory-safety bugs are common, and now a major source of exploits. In particular, a use-after-free bug occurs when an object is erroneously deallocated, whilst pointers to it remain active in memory, and those (dangling) pointers are later used to access the object. An attacker can reallocate the memory area backing an erroneously freed object, then overwrite its contents, injecting carefully chosen data into the host program, thus altering its execution and achieving privilege escalation. We present MineSweeper, a system to mitigate use-after-free vulnerabilities by retaining freed allocations in a quarantine, until no pointers to them remain in program memory, thus preventing their reallocation until it is safe. MineSweeper performs efficient linear sweeps of memory to identify quarantined items that have no dangling pointers to them, and thus can be safely reallocated. This allows MineSweeper to be significantly more efficient than previous transitive marking procedure techniques. MineSweeper, attached to JeMalloc, improves security at an acceptable overhead in memory footprint (11.1% on average) and an execution-time cost of only 5.4% (geometric mean for SPEC CPU2006), with 9.6% additional threaded CPU usage. These figures considerably improve on the state-of-the-art for non-probabilistic drop-in temporal-safety systems, and make MineSweeper the only such scheme suitable for deployment in real-world production environments. @InProceedings{ASPLOS22p212, author = {Márton Erdős and Sam Ainsworth and Timothy M. Jones}, title = {MineSweeper: A “Clean Sweep” for Drop-In Use-after-Free Prevention}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {212--225}, doi = {10.1145/3503222.3507712}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Allcock, Jonathan	ASPLOS '22: "Suppressing ZZ Crosstalk of ..." Suppressing ZZ Crosstalk of Quantum Computers through Pulse and Scheduling Co-Optimization Lei Xie, Jidong Zhai, ZhenXing Zhang, Jonathan Allcock, Shengyu Zhang, and Yi-Cong Zheng (Tsinghua University, China; Tencent Quantum Laboratory, China) Noise is a significant obstacle to quantum computing, and ZZ crosstalk is one of the most destructive types of noise affecting superconducting qubits. Previous approaches to suppressing ZZ crosstalk have mainly relied on specific chip design that can complicate chip fabrication and aggravate decoherence. To some extent, special chip design can be avoided by relying on pulse optimization to suppress ZZ crosstalk. However, existing approaches are non-scalable, as their required time and memory grow exponentially with the number of qubits involved. To address the above problems, we propose a scalable approach by co-optimizing pulses and scheduling. We optimize pulses to offer an ability to suppress ZZ crosstalk surrounding a gate, and then design scheduling strategies to exploit this ability and achieve suppression across the whole circuit. A main advantage of such co-optimization is that it does not require special hardware support. Besides, we implement our approach as a general framework that is compatible with different pulse optimization methods. We have conducted extensive evaluations by simulation and on a real quantum computer. Simulation results show that our proposal can improve the fidelity of quantum computing on 4∼12 qubits by up to 81× (11× on average). Ramsey experiments on a real quantum computer also demonstrate that our method can eliminate the effect of ZZ crosstalk to a great extent. @InProceedings{ASPLOS22p499, author = {Lei Xie and Jidong Zhai and ZhenXing Zhang and Jonathan Allcock and Shengyu Zhang and Yi-Cong Zheng}, title = {Suppressing ZZ Crosstalk of Quantum Computers through Pulse and Scheduling Co-Optimization}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {499--513}, doi = {10.1145/3503222.3507761}, year = {2022}, } Publisher's Version
	Almadhoun Alserr, Nour	ASPLOS '22: "GenStore: A High-Performance ..." GenStore: A High-Performance In-Storage Processing System for Genome Sequence Analysis Nika Mansouri Ghiasi, Jisung Park, Harun Mustafa, Jeremie Kim, Ataberk Olgun, Arvid Gollwitzer, Damla Senol Cali, Can Firtina, Haiyu Mao, Nour Almadhoun Alserr, Rachata Ausavarungnirun, Nandita Vijaykumar, Mohammed Alser, and Onur Mutlu (ETH Zurich, Switzerland; Bionano Genomics, USA; King Mongkut's University of Technology North Bangkok, Thailand; University of Toronto, Canada) Read mapping is a fundamental step in many genomics applications. It is used to identify potential matches and differences between fragments (called reads) of a sequenced genome and an already known genome (called a reference genome). Read mapping is costly because it needs to perform approximate string matching (ASM) on large amounts of data. To address the computational challenges in genome analysis, many prior works propose various approaches such as accurate filters that select the reads within a dataset of genomic reads (called a read set) that must undergo expensive computation, efficient heuristics, and hardware acceleration. While effective at reducing the amount of expensive computation, all such approaches still require the costly movement of a large amount of data from storage to the rest of the system, which can significantly lower the end-to-end performance of read mapping in conventional and emerging genomics systems. We propose GenStore, the first in-storage processing system designed for genome sequence analysis that greatly reduces both data movement and computational overheads of genome sequence analysis by exploiting low-cost and accurate in-storage filters. GenStore leverages hardware/software co-design to address the challenges of in-storage processing, supporting reads with 1) different properties such as read lengths and error rates, which highly depend on the sequencing technology, and 2) different degrees of genetic variation compared to the reference genome, which highly depends on the genomes that are being compared. Through rigorous analysis of read mapping processes of reads with different properties and degrees of genetic variation, we meticulously design low-cost hardware accelerators and data/computation flows inside a NAND flash-based solid-state drive (SSD). Our evaluation using a wide range of real genomic datasets shows that GenStore, when implemented in three modern NAND flash-based SSDs, significantly improves the read mapping performance of state-of-the-art software (hardware) baselines by 2.07-6.05× (1.52-3.32×) for read sets with high similarity to the reference genome and 1.45-33.63× (2.70-19.2×) for read sets with low similarity to the reference genome. @InProceedings{ASPLOS22p635, author = {Nika Mansouri Ghiasi and Jisung Park and Harun Mustafa and Jeremie Kim and Ataberk Olgun and Arvid Gollwitzer and Damla Senol Cali and Can Firtina and Haiyu Mao and Nour Almadhoun Alserr and Rachata Ausavarungnirun and Nandita Vijaykumar and Mohammed Alser and Onur Mutlu}, title = {GenStore: A High-Performance In-Storage Processing System for Genome Sequence Analysis}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {635--654}, doi = {10.1145/3503222.3507702}, year = {2022}, } Publisher's Version
	Alonso, Gustavo	ASPLOS '22: "Enzian: An Open, General, ..." Enzian: An Open, General, CPU/FPGA Platform for Systems Software Research David Cock, Abishek Ramdas, Daniel Schwyn, Michael Giardino, Adam Turowski, Zhenhao He, Nora Hossle, Dario Korolija, Melissa Licciardello, Kristina Martsenko, Reto Achermann, Gustavo Alonso, and Timothy Roscoe (ETH Zurich, Switzerland; University of British Columbia, Canada) Hybrid computing platforms, comprising CPU cores and FPGA logic, are increasingly used for accelerating data-intensive workloads in cloud deployments, and are a growing topic of interest in systems research. However, from a research perspective, existing hardware platforms are limited: they are often optimized for concrete, narrow use-cases and, therefore lack the flexibility needed to explore other applications and configurations. We show that a research group can design and build a more general, open, and affordable hardware platform for hybrid systems research. The platform, Enzian, is capable of duplicating the functionality of existing CPU/FPGA systems with comparable performance but in an open, flexible system. It couples a large FPGA with a server-class CPU in an asymmetric cache-coherent NUMA system. Enzian also enables research not possible with existing hybrid platforms, through explicit access to coherence messages, extensive thermal and power instrumentation, and an open, programmable baseboard management processor. Enzian is already being used in multiple projects, is open source (both hardware and software), and available for remote use. We present the design principles of Enzian, the challenges in building it, and evaluate it with a range of existing research use-cases alongside other, more specialized platforms, as well as demonstrating research not possible on existing platforms. @InProceedings{ASPLOS22p434, author = {David Cock and Abishek Ramdas and Daniel Schwyn and Michael Giardino and Adam Turowski and Zhenhao He and Nora Hossle and Dario Korolija and Melissa Licciardello and Kristina Martsenko and Reto Achermann and Gustavo Alonso and Timothy Roscoe}, title = {Enzian: An Open, General, CPU/FPGA Platform for Systems Software Research}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {434--451}, doi = {10.1145/3503222.3507742}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Alser, Mohammed	ASPLOS '22: "GenStore: A High-Performance ..." GenStore: A High-Performance In-Storage Processing System for Genome Sequence Analysis Nika Mansouri Ghiasi, Jisung Park, Harun Mustafa, Jeremie Kim, Ataberk Olgun, Arvid Gollwitzer, Damla Senol Cali, Can Firtina, Haiyu Mao, Nour Almadhoun Alserr, Rachata Ausavarungnirun, Nandita Vijaykumar, Mohammed Alser, and Onur Mutlu (ETH Zurich, Switzerland; Bionano Genomics, USA; King Mongkut's University of Technology North Bangkok, Thailand; University of Toronto, Canada) Read mapping is a fundamental step in many genomics applications. It is used to identify potential matches and differences between fragments (called reads) of a sequenced genome and an already known genome (called a reference genome). Read mapping is costly because it needs to perform approximate string matching (ASM) on large amounts of data. To address the computational challenges in genome analysis, many prior works propose various approaches such as accurate filters that select the reads within a dataset of genomic reads (called a read set) that must undergo expensive computation, efficient heuristics, and hardware acceleration. While effective at reducing the amount of expensive computation, all such approaches still require the costly movement of a large amount of data from storage to the rest of the system, which can significantly lower the end-to-end performance of read mapping in conventional and emerging genomics systems. We propose GenStore, the first in-storage processing system designed for genome sequence analysis that greatly reduces both data movement and computational overheads of genome sequence analysis by exploiting low-cost and accurate in-storage filters. GenStore leverages hardware/software co-design to address the challenges of in-storage processing, supporting reads with 1) different properties such as read lengths and error rates, which highly depend on the sequencing technology, and 2) different degrees of genetic variation compared to the reference genome, which highly depends on the genomes that are being compared. Through rigorous analysis of read mapping processes of reads with different properties and degrees of genetic variation, we meticulously design low-cost hardware accelerators and data/computation flows inside a NAND flash-based solid-state drive (SSD). Our evaluation using a wide range of real genomic datasets shows that GenStore, when implemented in three modern NAND flash-based SSDs, significantly improves the read mapping performance of state-of-the-art software (hardware) baselines by 2.07-6.05× (1.52-3.32×) for read sets with high similarity to the reference genome and 1.45-33.63× (2.70-19.2×) for read sets with low similarity to the reference genome. @InProceedings{ASPLOS22p635, author = {Nika Mansouri Ghiasi and Jisung Park and Harun Mustafa and Jeremie Kim and Ataberk Olgun and Arvid Gollwitzer and Damla Senol Cali and Can Firtina and Haiyu Mao and Nour Almadhoun Alserr and Rachata Ausavarungnirun and Nandita Vijaykumar and Mohammed Alser and Onur Mutlu}, title = {GenStore: A High-Performance In-Storage Processing System for Genome Sequence Analysis}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {635--654}, doi = {10.1145/3503222.3507702}, year = {2022}, } Publisher's Version
	Alston, Marc	ASPLOS '22: "PLD: Fast FPGA Compilation ..." PLD: Fast FPGA Compilation to Make Reconfigurable Acceleration Compatible with Modern Incremental Refinement Software Development Yuanlong Xiao, Eric Micallef, Andrew Butt, Matthew Hofmann, Marc Alston, Matthew Goldsmith, Andrew Merczynski-Hait, and André DeHon (University of Pennsylvania, USA) FPGA-based accelerators are demonstrating significant absolute performance and energy efficiency compared with general-purpose CPUs. While FPGA computations can now be described in standard, programming languages, like C, development for FPGAs accelerators remains tedious and inaccessible to modern software engineers. Slow compiles (potentially taking tens of hours) inhibit the rapid, incremental refinement of designs that is the hallmark of modern software engineering. To address this issue, we introduce separate compilation and linkage into the FPGA design flow, providing faster design turns more familiar to software development. To realize this flow, we provide abstractions, compiler options, and compiler flow that allow the same C source code to be compiled to processor cores in seconds and to FPGA regions in minutes, providing the missing -O0 and -O1 options familiar in software development. This raises the FPGA programming level and standardizes the programming experience, bringing FPGA-based accelerators into a more familiar software platform ecosystem for software engineers. @InProceedings{ASPLOS22p933, author = {Yuanlong Xiao and Eric Micallef and Andrew Butt and Matthew Hofmann and Marc Alston and Matthew Goldsmith and Andrew Merczynski-Hait and André DeHon}, title = {PLD: Fast FPGA Compilation to Make Reconfigurable Acceleration Compatible with Modern Incremental Refinement Software Development}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {933--945}, doi = {10.1145/3503222.3507740}, year = {2022}, } Publisher's Version
	Antichi, Gianni	ASPLOS '22: "Domain Specific Run Time Optimization ..." Domain Specific Run Time Optimization for Software Data Planes Sebastiano Miano, Alireza Sanaee, Fulvio Risso, Gábor Rétvári, and Gianni Antichi (Queen Mary University of London, UK; Politecnico di Torino, Italy; Budapest University of Technology and Economics, Hungary) State-of-the-art approaches to design, develop and optimize software packet-processing programs are based on static compilation: the compiler's input is a description of the forwarding plane semantics and the output is a binary that can accommodate any control plane configuration or input traffic. In this paper, we demonstrate that tracking control plane actions and packet-level traffic dynamics at run time opens up new opportunities for code specialization. We present Morpheus, a system working alongside static compilers that continuously optimizes the targeted networking code. We introduce a number of new techniques, from static code analysis to adaptive code instrumentation, and we implement a toolbox of domain specific optimizations that are not restricted to a specific data plane framework or programming language. We apply Morpheus to several eBPF and DPDK programs including Katran, Facebook's production-grade load balancer. We compare Morpheus against state-of-the-art optimization frameworks and show that it can bring up to 2x throughput improvement, while halving the 99th percentile latency. @InProceedings{ASPLOS22p1148, author = {Sebastiano Miano and Alireza Sanaee and Fulvio Risso and Gábor Rétvári and Gianni Antichi}, title = {Domain Specific Run Time Optimization for Software Data Planes}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1148--1164}, doi = {10.1145/3503222.3507769}, year = {2022}, } Publisher's Version Published Artifact Info Artifacts Available Artifacts Functional Results Reproduced
	Arya, Kapil	ASPLOS '22: "Memory-Harvesting VMs in Cloud ..." Memory-Harvesting VMs in Cloud Platforms Alexander Fuerst, Stanko Novaković, Íñigo Goiri, Gohar Irfan Chaudhry, Prateek Sharma, Kapil Arya, Kevin Broas, Eugene Bak, Mehmet Iyigun, and Ricardo Bianchini (Indiana University, USA; Microsoft Research, USA; Microsoft Azure, USA) loud platforms monetize their spare capacity by renting “Spot” virtual machines (VMs) that can be evicted in favor of higher-priority VMs. Recent work has shown that resource-harvesting VMs are more effective at exploiting spare capacity than Spot VMs, while also reducing the number of evictions. However, the prior work focused on harvesting CPU cores while keeping memory size fixed. This wastes a substantial monetization opportunity and may even limit the ability of harvesting VMs to leverage spare cores. Thus, in this paper, we explore memory harvesting and its challenges in real cloud platforms, namely its impact on VM creation time, NUMA spanning, and page fragmentation. We start by characterizing the amount and dynamics of the spare memory in Azure. We then design and implement memory-harvesting VMs (MHVMs), introducing new techniques for memory buffering, batching, and pre-reclamation. To demonstrate the use of MHVMs, we also extend a popular cluster scheduling framework (Hadoop) and a FaaS platform to adapt to them. Our main results show that (1) there is plenty of scope for memory harvesting in real platforms; (2) MHVMs are effective at mitigating the negative impacts of harvesting; and (3) our extensions of Hadoop and FaaS successfully hide the MHVMs’ varying memory size from the users’ data-processing jobs and functions. We conclude that memory harvesting has great potential for practical deployment and users can save up to 93% of their costs when running workloads on MHVMs. @InProceedings{ASPLOS22p583, author = {Alexander Fuerst and Stanko Novaković and Íñigo Goiri and Gohar Irfan Chaudhry and Prateek Sharma and Kapil Arya and Kevin Broas and Eugene Bak and Mehmet Iyigun and Ricardo Bianchini}, title = {Memory-Harvesting VMs in Cloud Platforms}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {583--594}, doi = {10.1145/3503222.3507725}, year = {2022}, } Publisher's Version
	Asmussen, Nils	ASPLOS '22: "Efficient and Scalable Core ..." Efficient and Scalable Core Multiplexing with M³v Nils Asmussen, Sebastian Haas, Carsten Weinhold, Till Miemietz, and Michael Roitzsch (Barkhausen Institut, Germany) The M³ system (ASPLOS ’16) proposed a hardware/software co-design that simplifies integration between general-purpose cores and special-purpose accelerators, allowing users to easily utilize them in a unified manner. M³ is a tiled architecture, whose tiles (cores and accelerators) are partitioned between applications, such that each tile is dedicated to its own application. The M³x system (ATC ’19) extended M³ by trading off some isolation to enable coarse-grained multiplexing of tiles among multiple applications. With M³x, if source tile t₁ runs code of application p and sends a message m to destination tile t₂ while t₂ is currently not associated with p, then m is forwarded to the right place through a “slow path”, via some special OS tile. In this paper, we present M³v, which extends M³x by further trading off some isolation between applications to support “fast path” communication that does not require the said OS tile’s involvement. Thus, with M³v, a tile can be efficiently multiplexed between applications provided it is a general-purpose core. M³v achieves this goal by 1) adding a local multiplexer to each such core, and by 2) virtualizing the core’s hardware component responsible for cross-tile communications. We prototype M³v using RISC-V cores on an FPGA platform and show that it significantly outperforms M³x and may achieve competitive performance to Linux. @InProceedings{ASPLOS22p452, author = {Nils Asmussen and Sebastian Haas and Carsten Weinhold and Till Miemietz and Michael Roitzsch}, title = {Efficient and Scalable Core Multiplexing with M³v}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {452--466}, doi = {10.1145/3503222.3507741}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Ausavarungnirun, Rachata	ASPLOS '22: "GenStore: A High-Performance ..." GenStore: A High-Performance In-Storage Processing System for Genome Sequence Analysis Nika Mansouri Ghiasi, Jisung Park, Harun Mustafa, Jeremie Kim, Ataberk Olgun, Arvid Gollwitzer, Damla Senol Cali, Can Firtina, Haiyu Mao, Nour Almadhoun Alserr, Rachata Ausavarungnirun, Nandita Vijaykumar, Mohammed Alser, and Onur Mutlu (ETH Zurich, Switzerland; Bionano Genomics, USA; King Mongkut's University of Technology North Bangkok, Thailand; University of Toronto, Canada) Read mapping is a fundamental step in many genomics applications. It is used to identify potential matches and differences between fragments (called reads) of a sequenced genome and an already known genome (called a reference genome). Read mapping is costly because it needs to perform approximate string matching (ASM) on large amounts of data. To address the computational challenges in genome analysis, many prior works propose various approaches such as accurate filters that select the reads within a dataset of genomic reads (called a read set) that must undergo expensive computation, efficient heuristics, and hardware acceleration. While effective at reducing the amount of expensive computation, all such approaches still require the costly movement of a large amount of data from storage to the rest of the system, which can significantly lower the end-to-end performance of read mapping in conventional and emerging genomics systems. We propose GenStore, the first in-storage processing system designed for genome sequence analysis that greatly reduces both data movement and computational overheads of genome sequence analysis by exploiting low-cost and accurate in-storage filters. GenStore leverages hardware/software co-design to address the challenges of in-storage processing, supporting reads with 1) different properties such as read lengths and error rates, which highly depend on the sequencing technology, and 2) different degrees of genetic variation compared to the reference genome, which highly depends on the genomes that are being compared. Through rigorous analysis of read mapping processes of reads with different properties and degrees of genetic variation, we meticulously design low-cost hardware accelerators and data/computation flows inside a NAND flash-based solid-state drive (SSD). Our evaluation using a wide range of real genomic datasets shows that GenStore, when implemented in three modern NAND flash-based SSDs, significantly improves the read mapping performance of state-of-the-art software (hardware) baselines by 2.07-6.05× (1.52-3.32×) for read sets with high similarity to the reference genome and 1.45-33.63× (2.70-19.2×) for read sets with low similarity to the reference genome. @InProceedings{ASPLOS22p635, author = {Nika Mansouri Ghiasi and Jisung Park and Harun Mustafa and Jeremie Kim and Ataberk Olgun and Arvid Gollwitzer and Damla Senol Cali and Can Firtina and Haiyu Mao and Nour Almadhoun Alserr and Rachata Ausavarungnirun and Nandita Vijaykumar and Mohammed Alser and Onur Mutlu}, title = {GenStore: A High-Performance In-Storage Processing System for Genome Sequence Analysis}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {635--654}, doi = {10.1145/3503222.3507702}, year = {2022}, } Publisher's Version
	Ayanzadeh, Ramin	ASPLOS '22: "HAMMER: Boosting Fidelity ..." HAMMER: Boosting Fidelity of Noisy Quantum Circuits by Exploiting Hamming Behavior of Erroneous Outcomes Swamit Tannu, Poulami Das, Ramin Ayanzadeh, and Moinuddin Qureshi (University of Wisconsin-Madison, USA; Georgia Institute of Technology, USA) Quantum computers with hundreds of qubits will be available soon. Unfortunately, high device error-rates pose a significant challenge in using these near-term quantum systems to power real-world applications. Executing a program on existing quantum systems generates both correct and incorrect outcomes, but often, the output distribution is too noisy to distinguish between them. In this paper, we show that erroneous outcomes are not arbitrary but exhibit a well-defined structure when represented in the Hamming space. Our experiments on IBM and Google quantum computers show that the most frequent erroneous outcomes are more likely to be close in the Hamming space to the correct outcome. We exploit this behavior to improve the ability to infer the correct outcome. We propose Hamming Reconstruction (HAMMER), a post-processing technique that leverages the observation of Hamming behavior to reconstruct the noisy output distribution, such that the resulting distribution has higher fidelity. We evaluate HAMMER using experimental data from Google and IBM quantum computers with more than 500 unique quantum circuits and obtain an average improvement of 1.37x in the quality of solution. On Google’s publicly available QAOA datasets, we show that HAMMER sharpens the gradients on the cost function landscape. @InProceedings{ASPLOS22p529, author = {Swamit Tannu and Poulami Das and Ramin Ayanzadeh and Moinuddin Qureshi}, title = {HAMMER: Boosting Fidelity of Noisy Quantum Circuits by Exploiting Hamming Behavior of Erroneous Outcomes}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {529--540}, doi = {10.1145/3503222.3507703}, year = {2022}, } Publisher's Version
	Ayers, Grant	ASPLOS '22: "CRISP: Critical Slice Prefetching ..." CRISP: Critical Slice Prefetching Heiner Litz, Grant Ayers, and Parthasarathy Ranganathan (University of California at Santa Cruz, USA; Google, USA) The high access latency of DRAM continues to be a performance challenge for contemporary microprocessor systems. Prefetching is a well-established technique to address this problem, however, existing implemented designs fail to provide any performance benefits in the presence of irregular memory access patterns. The hardware complexity of prior techniques that can predict irregular memory accesses such as runahead execution has proven untenable for implementation in real hardware. We propose a lightweight mechanism to hide the high latency of irregular memory access patterns by leveraging criticality-based scheduling. In particular, our technique executes delinquent loads and their load slices as early as possible, hiding a significant fraction of their latency. Furthermore, we observe that the latency induced by branch mispredictions and other high latency instructions can be hidden with a similar approach. Our proposal only requires minimal hardware modifications by performing memory access classification, load and branch slice extraction, as well as priority analysis exclusively in software. As a result, our technique is feasible to implement, introducing only a simple new instruction prefix while requiring minimal modifications of the instruction scheduler. Our technique increases the IPC of memory-latency-bound applications by up to 38% and by 8.4% on average. @InProceedings{ASPLOS22p300, author = {Heiner Litz and Grant Ayers and Parthasarathy Ranganathan}, title = {CRISP: Critical Slice Prefetching}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {300--313}, doi = {10.1145/3503222.3507745}, year = {2022}, } Publisher's Version
	Bacik, Josef	ASPLOS '22: "IOCost: Block IO Control for ..." IOCost: Block IO Control for Containers in Datacenters Tejun Heo, Dan Schatzberg, Andrew Newell, Song Liu, Saravanan Dhakshinamurthy, Iyswarya Narayanan, Josef Bacik, Chris Mason, Chunqiang Tang, and Dimitrios Skarlatos (Meta, USA; Carnegie Mellon University, USA) Resource isolation is a fundamental requirement in datacenter environments. However, our production experience in Meta’s large-scale datacenters shows that existing IO control mechanisms for block storage are inadequate in containerized environments. IO control needs to provide proportional resources to containers while taking into account the hardware heterogeneity of storage devices and the idiosyncrasies of the workloads deployed in datacenters. The speed of modern SSDs requires IO control to execute with low-overheads. Furthermore, IO control should strive for work conservation, take into account the interactions with the memory management subsystem, and avoid priority inversions that lead to isolation failures. To address these challenges, this paper presents IOCost, an IO control solution that is designed for containerized environments and provides scalable, work-conserving, and low-overhead IO control for heterogeneous storage devices and diverse workloads in datacenters. IOCost performs offline profiling to build a device model and uses it to estimate device occupancy of each IO request. To minimize runtime overhead, it separates IO control into a fast per-IO issue path and a slower periodic planning path. A novel work-conserving budget donation algorithm enables containers to dynamically share unused budget. We have deployed IOCost across the entirety of Meta’s datacenters comprised of millions of ma- chines, upstreamed IOCost to the Linux kernel, and open-sourced our device-profiling tools. IOCost has been running in production for two years, providing IO control for Meta’s fleet. We describe the design of IOCost and share our experience deploying it at scale. @InProceedings{ASPLOS22p595, author = {Tejun Heo and Dan Schatzberg and Andrew Newell and Song Liu and Saravanan Dhakshinamurthy and Iyswarya Narayanan and Josef Bacik and Chris Mason and Chunqiang Tang and Dimitrios Skarlatos}, title = {IOCost: Block IO Control for Containers in Datacenters}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {595--608}, doi = {10.1145/3503222.3507727}, year = {2022}, } Publisher's Version
	Bădoiu, Vlad-Andrei	ASPLOS '22: "FlexOS: Towards Flexible OS ..." FlexOS: Towards Flexible OS Isolation Hugo Lefeuvre, Vlad-Andrei Bădoiu, Alexander Jung, Stefan Lucian Teodorescu, Sebastian Rauch, Felipe Huici, Costin Raiciu, and Pierre Olivier (University of Manchester, UK; University Politehnica of Bucharest, Romania; Lancaster University, UK; unikraft.io, Germany; KIT, Germany; NEC Laboratories, Germany; Correct Networks, Romania) At design time, modern operating systems are locked in a specific safety and isolation strategy that mixes one or more hardware/software protection mechanisms (e.g. user/kernel separation); revisiting these choices after deployment requires a major refactoring effort. This rigid approach shows its limits given the wide variety of modern applications' safety/performance requirements, when new hardware isolation mechanisms are rolled out, or when existing ones break. We present FlexOS, a novel OS allowing users to easily specialize the safety and isolation strategy of an OS at compilation/deployment time instead of design time. This modular LibOS is composed of fine-grained components that can be isolated via a range of hardware protection mechanisms with various data sharing strategies and additional software hardening. The OS ships with an exploration technique helping the user navigate the vast safety/performance design space it unlocks. We implement a prototype of the system and demonstrate, for several applications (Redis/Nginx/SQLite), FlexOS' vast configuration space as well as the efficiency of the exploration technique: we evaluate 80 FlexOS configurations for Redis and show how that space can be probabilistically subset to the 5 safest ones under a given performance budget. We also show that, under equivalent configurations, FlexOS performs similarly or better than existing solutions which use fixed safety configurations. @InProceedings{ASPLOS22p467, author = {Hugo Lefeuvre and Vlad-Andrei Bădoiu and Alexander Jung and Stefan Lucian Teodorescu and Sebastian Rauch and Felipe Huici and Costin Raiciu and Pierre Olivier}, title = {FlexOS: Towards Flexible OS Isolation}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {467--482}, doi = {10.1145/3503222.3507759}, year = {2022}, } Publisher's Version Published Artifact Info Artifacts Available Artifacts Functional Results Reproduced
	Bagué, Roger Figueras	ASPLOS '22: "BiSon-e: A Lightweight and ..." BiSon-e: A Lightweight and High-Performance Accelerator for Narrow Integer Linear Algebra Computing on the Edge Enrico Reggiani, Cristóbal Ramírez Lazo, Roger Figueras Bagué, Adrián Cristal, Mauro Olivieri, and Osman Sabri Unsal (Polytechnic University of Catalonia, Spain; Barcelona Supercomputing Center, Spain; Sapienza University of Rome, Italy) Linear algebra computational kernels based on byte and sub-byte integer data formats are at the base of many classes of applications, ranging from Deep Learning to Pattern Matching. Porting the computation of these applications from cloud to edge and mobile devices would enable significant improvements in terms of security, safety, and energy efficiency. However, despite their low memory and energy demands, their intrinsically high computational intensity makes the execution of these workloads challenging on highly resource-constrained devices. In this paper, we present BiSon-e, a novel RISC-V based architecture that accelerates linear algebra kernels based on narrow integer computations on edge processors by performing Single Instruction Multiple Data (SIMD) operations on off-the-shelf scalar Functional Units (FUs). Our novel architecture is built upon the binary segmentation technique, which allows to significantly reduce the memory footprint and the arithmetic intensity of linear algebra kernels requiring narrow data sizes. We integrate BiSon-e into a complete System-on-Chip (SoC) based on RISC-V, synthesized and Place&Routed in 65nm and 22nm technologies, introducing a negligible 0.07% area overhead with respect to the baseline architecture. Our experimental evaluation shows that, when computing the Convolution and Fully-Connected layers of the AlexNet and VGG-16 Convolutional Neural Networks (CNNs) with 8-, 4-, and 2-bit, our solution gains up to 5.6×, 13.9× and 24× in execution time compared to the scalar implementation of a single RISC-V core, and improves the energy efficiency of string matching tasks by 5× when compared to a RISC-V-based Vector Processing Unit (VPU). @InProceedings{ASPLOS22p56, author = {Enrico Reggiani and Cristóbal Ramírez Lazo and Roger Figueras Bagué and Adrián Cristal and Mauro Olivieri and Osman Sabri Unsal}, title = {BiSon-e: A Lightweight and High-Performance Accelerator for Narrow Integer Linear Algebra Computing on the Edge}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {56--69}, doi = {10.1145/3503222.3507746}, year = {2022}, } Publisher's Version
	Bai, Jia-Ju	ASPLOS '22: "Path-Sensitive and Alias-Aware ..." Path-Sensitive and Alias-Aware Typestate Analysis for Detecting OS Bugs Tuo Li, Jia-Ju Bai, Yulei Sui, and Shi-Min Hu (Tsinghua University, China; University of Technology Sydney, Australia) Operating system (OS) is the cornerstone for modern computer systems. It manages devices and provides fundamental service for user-level applications. Thus, detecting bugs in OSes is important to improve reliability and security of computer systems. Static typestate analysis is a common technique for detecting different types of bugs, but it is often inaccurate or unscalable for large-size OS code, due to imprecision of identifying alias relationships as well as high costs of typestate tracking and path-feasibility validation. In this paper, we present PATA, a novel path-sensitive and aliasaware typestate analysis framework to detect OS bugs. To improve the precision of identifying alias relationships in OS code, PATA performs a path-based alias analysis based on control-flow paths and access paths. With these alias relationships, PATA reduces the costs of typestate tracking and path-feasibility validation, to boost the efficiency of path-sensitive typestate analysis for bug detection. We have evaluated PATA on the Linux kernel and three popular IoT OSes (Zephyr, RIOT and TencentOS-tiny) to detect three common types of bugs (null-pointer dereferences, uninitialized variable accesses and memory leaks). PATA finds 574 real bugs with a false positive rate of 28%. 206 of these bugs have been confirmed by the developers of the four OSes.We also compare PATA to seven state-of-the-art static approaches (Cppcheck, Coccinelle, Smatch,CSA, Infer, Saber and SVF). PATA finds many real bugs missed by them, with a lower false positive rate. @InProceedings{ASPLOS22p859, author = {Tuo Li and Jia-Ju Bai and Yulei Sui and Shi-Min Hu}, title = {Path-Sensitive and Alias-Aware Typestate Analysis for Detecting OS Bugs}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {859--872}, doi = {10.1145/3503222.3507770}, year = {2022}, } Publisher's Version
	Bak, Eugene	ASPLOS '22: "Memory-Harvesting VMs in Cloud ..." Memory-Harvesting VMs in Cloud Platforms Alexander Fuerst, Stanko Novaković, Íñigo Goiri, Gohar Irfan Chaudhry, Prateek Sharma, Kapil Arya, Kevin Broas, Eugene Bak, Mehmet Iyigun, and Ricardo Bianchini (Indiana University, USA; Microsoft Research, USA; Microsoft Azure, USA) loud platforms monetize their spare capacity by renting “Spot” virtual machines (VMs) that can be evicted in favor of higher-priority VMs. Recent work has shown that resource-harvesting VMs are more effective at exploiting spare capacity than Spot VMs, while also reducing the number of evictions. However, the prior work focused on harvesting CPU cores while keeping memory size fixed. This wastes a substantial monetization opportunity and may even limit the ability of harvesting VMs to leverage spare cores. Thus, in this paper, we explore memory harvesting and its challenges in real cloud platforms, namely its impact on VM creation time, NUMA spanning, and page fragmentation. We start by characterizing the amount and dynamics of the spare memory in Azure. We then design and implement memory-harvesting VMs (MHVMs), introducing new techniques for memory buffering, batching, and pre-reclamation. To demonstrate the use of MHVMs, we also extend a popular cluster scheduling framework (Hadoop) and a FaaS platform to adapt to them. Our main results show that (1) there is plenty of scope for memory harvesting in real platforms; (2) MHVMs are effective at mitigating the negative impacts of harvesting; and (3) our extensions of Hadoop and FaaS successfully hide the MHVMs’ varying memory size from the users’ data-processing jobs and functions. We conclude that memory harvesting has great potential for practical deployment and users can save up to 93% of their costs when running workloads on MHVMs. @InProceedings{ASPLOS22p583, author = {Alexander Fuerst and Stanko Novaković and Íñigo Goiri and Gohar Irfan Chaudhry and Prateek Sharma and Kapil Arya and Kevin Broas and Eugene Bak and Mehmet Iyigun and Ricardo Bianchini}, title = {Memory-Harvesting VMs in Cloud Platforms}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {583--594}, doi = {10.1145/3503222.3507725}, year = {2022}, } Publisher's Version
	Bandara, Thilini Kaushalya	ASPLOS '22: "REVAMP: A Systematic Framework ..." REVAMP: A Systematic Framework for Heterogeneous CGRA Realization Thilini Kaushalya Bandara, Dhananjaya Wijerathne, Tulika Mitra, and Li-Shiuan Peh (National University of Singapore, Singapore) Coarse-Grained Reconfigurable Architectures (CGRAs) provide an excellent balance between performance, energy efficiency, and flexibility. However, increasingly sophisticated applications, especially on the edge devices, demand even better energy efficiency for longer battery life. Most CGRAs adhere to a canonical structure where a homogeneous set of processing elements and memories communicate through a regular interconnect due to the simplicity of the design. Unfortunately, the homogeneity leads to substantial idle resources while mapping irregular applications and creates inefficiency. We plan to mitigate the inefficiency by systematically and judiciously introducing heterogeneity in CGRAs in tandem with appropriate compiler support. We propose REVAMP, an automated design space exploration framework that helps architects uncover and add pertinent heterogeneity to a diverse range of originally homogeneous CGRAs when fed with a suite of target applications. REVAMP explores a comprehensive set of optimizations encompassing compute, network, and memory heterogeneity, thereby converting a uniform CGRA into a more irregular architecture with improved energy efficiency. As CGRAs are inherently software scheduled, any micro-architectural optimizations need to be partnered with corresponding compiler support, which is challenging with heterogeneity. The REVAMP framework extends compiler support for efficient mapping of loop kernels on the derived heterogeneous CGRA architectures. We showcase REVAMP on three state-of-the-art homogeneous CGRAs, demonstrating how REVAMP derives a heterogeneous variant of each homogeneous architecture, with its corresponding compiler optimizations. Our results show that the derived heterogeneous architectures achieve up to 52.4% power reduction, 38.1% area reduction, and 36% average energy reduction over the corresponding homogeneous versions with minimal performance impact for the selected kernel suite. @InProceedings{ASPLOS22p918, author = {Thilini Kaushalya Bandara and Dhananjaya Wijerathne and Tulika Mitra and Li-Shiuan Peh}, title = {REVAMP: A Systematic Framework for Heterogeneous CGRA Realization}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {918--932}, doi = {10.1145/3503222.3507772}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Bao, Tiffany	ASPLOS '22: "ViK: Practical Mitigation ..." ViK: Practical Mitigation of Temporal Memory Safety Violations through Object ID Inspection Haehyun Cho, Jinbum Park, Adam Oest, Tiffany Bao, Ruoyu Wang, Yan Shoshitaishvili, Adam Doupé, and Gail-Joon Ahn (Soongsil University, South Korea; Samsung Research, South Korea; PayPal, USA; Arizona State University, USA; Samsung Research, USA) Temporal memory safety violations, such as use-after-free (UAF) vulnerabilities, are a critical security issue for software written in memory-unsafe languages such as C and C++. In this paper, we introduce ViK, a novel, lightweight, and widely applicable runtime defense that can protect both operating system (OS) kernels and user-space applications against temporal memory safety violations. ViK performs object ID inspection, where it assigns a random identifier to every allocated object and stores the identifier in the unused bits of the corresponding pointer. When the pointer is used, ViK inspects the value of a pointer before dereferencing, ensuring that the pointer still references the original object. To the best of our knowledge, this is the first mitigation against temporal memory safety violations that scales to OS kernels. We evaluated the software prototype of ViK on Android and Linux kernels and observed runtime overhead of around 20%. Also, we evaluated a hardware-assisted prototype of ViK on Android kernel, where the runtime overhead was as low as 2%. @InProceedings{ASPLOS22p271, author = {Haehyun Cho and Jinbum Park and Adam Oest and Tiffany Bao and Ruoyu Wang and Yan Shoshitaishvili and Adam Doupé and Gail-Joon Ahn}, title = {ViK: Practical Mitigation of Temporal Memory Safety Violations through Object ID Inspection}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {271--284}, doi = {10.1145/3503222.3507780}, year = {2022}, } Publisher's Version Artifacts Functional
	Basu, Arkaprava	ASPLOS '22: "GPM: Leveraging Persistent ..." GPM: Leveraging Persistent Memory from a GPU Shweta Pandey, Aditya K Kamath, and Arkaprava Basu (IISc Bangalore, India) The GPU is a key computing platform for many application domains. While the new non-volatile memory technology has brought the promise of byte-addressable persistence (a.k.a., persistent memory, or PM) to CPU applications, the same, unfortunately, is beyond the reach of GPU programs. We take three key steps toward enabling GPU programs to access PM directly. First, enable direct access to PM from within a GPU kernel without needing to modify the hardware. Next, we demonstrate three classes of GPU-accelerated applications that benefit from PM. In the process, we create a workload suite with nine such applications. We then create a GPU library, written in CUDA, to support logging, checkpointing, and primitives for native persistence for programmers to easily leverage PM. @InProceedings{ASPLOS22p142, author = {Shweta Pandey and Aditya K Kamath and Arkaprava Basu}, title = {GPM: Leveraging Persistent Memory from a GPU}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {142--156}, doi = {10.1145/3503222.3507758}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Bautista, Meriam Gay	ASPLOS '22: "Temporal and SFQ Pulse-Streams ..." Temporal and SFQ Pulse-Streams Encoding for Area-Efficient Superconducting Accelerators Patricia Gonzalez-Guerrero, Meriam Gay Bautista, Darren Lyles, and George Michelogiannakis (Lawrence Berkeley National Laboratory, USA) Superconducting technology is a prime candidate for the future of computing. However, current superconducting prototypes are limited to small-scale examples due to stringent area constraints and complex architectures inspired from voltage-level encoding in CMOS; this is at odds with the ps-wide Single Quantum Flux (SFQ) pulses used in superconductors to carry information. In this work, we propose a wave-pipelined Unary SFQ (U-SFQ) architecture that leverages the advantages of two data representations: pulse-streams and Race Logic (RL). We introduce novel building blocks such as multipliers, adders, and memory cells, which leverage the natural properties of SFQ pulses to mitigate area constraints. We then design and simulate three popular hardware accelerators: i) a Processing Element (PE), typically used in spatial architectures; ii) A dot-product-unit (DPU), one of the most popular accelerators in artificial neural networks and digital signal processing (DSP); and iii) A Finite Impulse Response (FIR) filter, a popular and computationally demanding DSP accelerator. The proposed U-SFQ building blocks require up to 200× fewer JJs compared to their SFQ binary counterparts, exposing an area-delay trade-off. This work mitigates the stringent area constraints of superconducting technology. @InProceedings{ASPLOS22p963, author = {Patricia Gonzalez-Guerrero and Meriam Gay Bautista and Darren Lyles and George Michelogiannakis}, title = {Temporal and SFQ Pulse-Streams Encoding for Area-Efficient Superconducting Accelerators}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {963--976}, doi = {10.1145/3503222.3507765}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available
	Berger, Daniel	ASPLOS '22: "SOL: Safe On-Node Learning ..." SOL: Safe On-Node Learning in Cloud Platforms Yawen Wang, Daniel Crankshaw, Neeraja J. Yadwadkar, Daniel Berger, Christos Kozyrakis, and Ricardo Bianchini (Stanford University, USA; Microsoft Research, USA; University of Texas at Austin, USA) Cloud platforms run many software agents on each server node. These agents manage all aspects of node operation, and in some cases frequently collect data and make decisions. Unfortunately, their behavior is typically based on pre-defined static heuristics or offline analysis; they do not leverage on-node machine learning (ML). In this paper, we first characterize the spectrum of node agents in Azure, and identify the classes of agents that are most likely to benefit from on-node ML. We then propose SOL, an extensible framework for designing ML-based agents that are safe and robust to the range of failure conditions that occur in production. SOL provides a simple API to agent developers and manages the scheduling and running of the agent-specific functions they write. We illustrate the use of SOL by implementing three ML-based agents that manage CPU cores, node power, and memory placement. Our experiments show that (1) ML substantially improves our agents, and (2) SOL ensures that agents operate safely under a variety of failure conditions. We conclude that ML-based agents show significant potential and that SOL can help build them. @InProceedings{ASPLOS22p622, author = {Yawen Wang and Daniel Crankshaw and Neeraja J. Yadwadkar and Daniel Berger and Christos Kozyrakis and Ricardo Bianchini}, title = {SOL: Safe On-Node Learning in Cloud Platforms}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {622--634}, doi = {10.1145/3503222.3507704}, year = {2022}, } Publisher's Version
	Bernat, Alex	ASPLOS '22: "CARAT CAKE: Replacing Paging ..." CARAT CAKE: Replacing Paging via Compiler/Kernel Cooperation Brian Suchy, Souradip Ghosh, Drew Kersnar, Siyuan Chai, Zhen Huang, Aaron Nelson, Michael Cuevas, Alex Bernat, Gaurav Chaudhary, Nikos Hardavellas, Simone Campanoni, and Peter Dinda (Northwestern University, USA) Virtual memory, specifically paging, is undergoing significant innovation due to being challenged by new demands from modern workloads. Recent work has demonstrated an alternative software only design that can result in simplified hardware requirements, even supporting purely physical addressing. While we have made the case for this Compiler- And Runtime-based Address Translation (CARAT) concept, its evaluation was based on a user-level prototype. We now report on incorporating CARAT into a kernel, forming Compiler- And Runtime-based Address Translation for CollAborative Kernel Environments (CARAT CAKE). In our implementation, a Linux-compatible x64 process abstraction can be based either on CARAT CAKE, or on a sophisticated paging implementation. Implementing CARAT CAKE involves kernel changes and compiler optimizations/transformations that must work on all code in the system, including kernel code. We evaluate CARAT CAKE in comparison with paging and find that CARAT CAKE is able to achieve the functionality of paging (protection, mapping, and movement properties) with minimal overhead. In turn, CARAT CAKE allows significant new benefits for systems including energy savings, larger L1 caches, and arbitrary granularity memory management. @InProceedings{ASPLOS22p98, author = {Brian Suchy and Souradip Ghosh and Drew Kersnar and Siyuan Chai and Zhen Huang and Aaron Nelson and Michael Cuevas and Alex Bernat and Gaurav Chaudhary and Nikos Hardavellas and Simone Campanoni and Peter Dinda}, title = {CARAT CAKE: Replacing Paging via Compiler/Kernel Cooperation}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {98--114}, doi = {10.1145/3503222.3507771}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available
	Bianchini, Ricardo	ASPLOS '22: "Memory-Harvesting VMs in Cloud ..." Memory-Harvesting VMs in Cloud Platforms Alexander Fuerst, Stanko Novaković, Íñigo Goiri, Gohar Irfan Chaudhry, Prateek Sharma, Kapil Arya, Kevin Broas, Eugene Bak, Mehmet Iyigun, and Ricardo Bianchini (Indiana University, USA; Microsoft Research, USA; Microsoft Azure, USA) loud platforms monetize their spare capacity by renting “Spot” virtual machines (VMs) that can be evicted in favor of higher-priority VMs. Recent work has shown that resource-harvesting VMs are more effective at exploiting spare capacity than Spot VMs, while also reducing the number of evictions. However, the prior work focused on harvesting CPU cores while keeping memory size fixed. This wastes a substantial monetization opportunity and may even limit the ability of harvesting VMs to leverage spare cores. Thus, in this paper, we explore memory harvesting and its challenges in real cloud platforms, namely its impact on VM creation time, NUMA spanning, and page fragmentation. We start by characterizing the amount and dynamics of the spare memory in Azure. We then design and implement memory-harvesting VMs (MHVMs), introducing new techniques for memory buffering, batching, and pre-reclamation. To demonstrate the use of MHVMs, we also extend a popular cluster scheduling framework (Hadoop) and a FaaS platform to adapt to them. Our main results show that (1) there is plenty of scope for memory harvesting in real platforms; (2) MHVMs are effective at mitigating the negative impacts of harvesting; and (3) our extensions of Hadoop and FaaS successfully hide the MHVMs’ varying memory size from the users’ data-processing jobs and functions. We conclude that memory harvesting has great potential for practical deployment and users can save up to 93% of their costs when running workloads on MHVMs. @InProceedings{ASPLOS22p583, author = {Alexander Fuerst and Stanko Novaković and Íñigo Goiri and Gohar Irfan Chaudhry and Prateek Sharma and Kapil Arya and Kevin Broas and Eugene Bak and Mehmet Iyigun and Ricardo Bianchini}, title = {Memory-Harvesting VMs in Cloud Platforms}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {583--594}, doi = {10.1145/3503222.3507725}, year = {2022}, } Publisher's Version ASPLOS '22: "SOL: Safe On-Node Learning ..." SOL: Safe On-Node Learning in Cloud Platforms Yawen Wang, Daniel Crankshaw, Neeraja J. Yadwadkar, Daniel Berger, Christos Kozyrakis, and Ricardo Bianchini (Stanford University, USA; Microsoft Research, USA; University of Texas at Austin, USA) Cloud platforms run many software agents on each server node. These agents manage all aspects of node operation, and in some cases frequently collect data and make decisions. Unfortunately, their behavior is typically based on pre-defined static heuristics or offline analysis; they do not leverage on-node machine learning (ML). In this paper, we first characterize the spectrum of node agents in Azure, and identify the classes of agents that are most likely to benefit from on-node ML. We then propose SOL, an extensible framework for designing ML-based agents that are safe and robust to the range of failure conditions that occur in production. SOL provides a simple API to agent developers and manages the scheduling and running of the agent-specific functions they write. We illustrate the use of SOL by implementing three ML-based agents that manage CPU cores, node power, and memory placement. Our experiments show that (1) ML substantially improves our agents, and (2) SOL ensures that agents operate safely under a variety of failure conditions. We conclude that ML-based agents show significant potential and that SOL can help build them. @InProceedings{ASPLOS22p622, author = {Yawen Wang and Daniel Crankshaw and Neeraja J. Yadwadkar and Daniel Berger and Christos Kozyrakis and Ricardo Bianchini}, title = {SOL: Safe On-Node Learning in Cloud Platforms}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {622--634}, doi = {10.1145/3503222.3507704}, year = {2022}, } Publisher's Version
	Black-Schaffer, David	ASPLOS '22: "Every Walk’s a Hit: Making ..." Every Walk’s a Hit: Making Page Walks Single-Access Cache Hits Chang Hyun Park, Ilias Vougioukas, Andreas Sandberg, and David Black-Schaffer (Uppsala University, Sweden; Arm Research, UK) As memory capacity has outstripped TLB coverage, large data applications suffer from frequent page table walks. We investigate two complementary techniques for addressing this cost: reducing the number of accesses required and reducing the latency of each access. The first approach is accomplished by opportunistically "flattening" the page table: merging two levels of traditional 4 KB page table nodes into a single 2 MB node, thereby reducing the table's depth and the number of indirections required to traverse it. The second is accomplished by biasing the cache replacement algorithm to keep page table entries during periods of high TLB miss rates, as these periods also see high data miss rates and are therefore more likely to benefit from having the smaller page table in the cache than to suffer from increased data cache misses. We evaluate these approaches for both native and virtualized systems and across a range of realistic memory fragmentation scenarios, describe the limited changes needed in our kernel implementation and hardware design, identify and address challenges related to self-referencing page tables and kernel memory allocation, and compare results across server and mobile systems using both academic and industrial simulators for robustness. We find that flattening does reduce the number of accesses required on a page walk (to 1.0), but its performance impact (+2.3%) is small due to Page Walker Caches (already 1.5 accesses). Prioritizing caching has a larger effect (+6.8%), and the combination improves performance by +9.2%. Flattening is more effective on virtualized systems (4.4 to 2.8 accesses, +7.1% performance), due to 2D page walks. By combining the two techniques we demonstrate a state-of-the-art +14.0% performance gain and -8.7% dynamic cache energy and -4.7% dynamic DRAM energy for virtualized execution with very simple hardware and software changes. @InProceedings{ASPLOS22p128, author = {Chang Hyun Park and Ilias Vougioukas and Andreas Sandberg and David Black-Schaffer}, title = {Every Walk’s a Hit: Making Page Walks Single-Access Cache Hits}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {128--141}, doi = {10.1145/3503222.3507718}, year = {2022}, } Publisher's Version
	Bodik, Rastislav	ASPLOS '22: "Tree Traversal Synthesis Using ..." Tree Traversal Synthesis Using Domain-Specific Symbolic Compilation Yanju Chen, Junrui Liu, Yu Feng, and Rastislav Bodik (University of California at Santa Barbara, USA; University of Washington, USA) Efficient computation on tree data structures is important in compilers, numeric computations, and web browser layout engines. Efficiency is achieved by statically scheduling the computation into a small number of tree traversals and by performing the traversals in parallel when possible. Manual design of such traversals leads to bugs, as observed in web browsers. Automatic schedulers avoid these bugs but they currently cannot explore a space of legal traversals, which prevents exploring the trade-offs between parallelism and minimizing the number of traversals. We describe Hecate, a synthesizer of tree traversals that can produce both serial and parallel traversals. A key feature is that the synthesizer is extensible by the programmer who can define a template for new kinds of traversals. Hecate is constructed as a solver-aided domain-specific language, meaning that the synthesizer is generated automatically by translating the tree traversal DSL to an SMT solver that synthesizes the traversals. We improve on the general-purpose solver-aided architecture with a scheduling-specific symbolic evaluation that maintains the engineering advantages solver-aided design but generates efficient ILP encoding that is much more efficient to solve than SMT constraints. On the set of Grafter problems, Hecate synthesizes traversals that trade off traversal fusion to exploit parallelism. Additionally, Hecate allows defining a tree data structure with an arbitrary number of children. Together, parallelism and data structure improvements accelerate the computation 2× on a tree rendering problem. Finally, Hecate’s domain-specific symbolic compilation accelerates synthesis 3× compared to the general-purpose compilation to an SMT solver; when scheduling a CSS engine traversal, this ILP-based synthesis executes orders of magnitude faster. @InProceedings{ASPLOS22p1030, author = {Yanju Chen and Junrui Liu and Yu Feng and Rastislav Bodik}, title = {Tree Traversal Synthesis Using Domain-Specific Symbolic Compilation}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1030--1042}, doi = {10.1145/3503222.3507751}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional
	Bourgeat, Thomas	ASPLOS '22: "DAGguise: Mitigating Memory ..." DAGguise: Mitigating Memory Timing Side Channels Peter W. Deutsch, Yuheng Yang, Thomas Bourgeat, Jules Drean, Joel S. Emer, and Mengjia Yan (Massachusetts Institute of Technology, USA; NVIDIA, USA) This paper studies the mitigation of memory timing side channels, where attackers utilize contention within DRAM controllers to infer a victim’s secrets. Already practical, this class of channels poses an important challenge to secure computing in shared memory environments. Existing state-of-the-art memory timing side channel mitigations have several key performance and security limitations. Prior schemes require onerous static bandwidth partitioning, extensive profiling phases, or simply fail to protect against attacks which exploit fine-grained timing and bank information. We present DAGguise, a defense mechanism which fully protects against memory timing side channels while allowing for dynamic traffic contention in order to achieve good performance. DAGguise utilizes a novel abstract memory access representation, the Directed Acyclic Request Graph (rDAG for short), to model memory access patterns which experience contention. DAGguise shapes a victim’s memory access patterns according to a publicly known rDAG obtained through a lightweight profiling stage, completely eliminating information leakage. We formally verify the security of DAGguise, proving that it maintains strong security guarantees. Moreover, by allowing dynamic traffic contention, DAGguise achieves a 12% overall system speedup relative to Fixed Service, which is the state-of-the-art mitigation mechanism, with up to a 20% relative speedup for co-located applications which do not require protection. We further claim that the principles of DAGguise can be generalized to protect against other types of scheduler-based timing side channels, such as those targeting on-chip networks, or functional units in SMT cores. @InProceedings{ASPLOS22p329, author = {Peter W. Deutsch and Yuheng Yang and Thomas Bourgeat and Jules Drean and Joel S. Emer and Mengjia Yan}, title = {DAGguise: Mitigating Memory Timing Side Channels}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {329--343}, doi = {10.1145/3503222.3507747}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Broas, Kevin	ASPLOS '22: "Memory-Harvesting VMs in Cloud ..." Memory-Harvesting VMs in Cloud Platforms Alexander Fuerst, Stanko Novaković, Íñigo Goiri, Gohar Irfan Chaudhry, Prateek Sharma, Kapil Arya, Kevin Broas, Eugene Bak, Mehmet Iyigun, and Ricardo Bianchini (Indiana University, USA; Microsoft Research, USA; Microsoft Azure, USA) loud platforms monetize their spare capacity by renting “Spot” virtual machines (VMs) that can be evicted in favor of higher-priority VMs. Recent work has shown that resource-harvesting VMs are more effective at exploiting spare capacity than Spot VMs, while also reducing the number of evictions. However, the prior work focused on harvesting CPU cores while keeping memory size fixed. This wastes a substantial monetization opportunity and may even limit the ability of harvesting VMs to leverage spare cores. Thus, in this paper, we explore memory harvesting and its challenges in real cloud platforms, namely its impact on VM creation time, NUMA spanning, and page fragmentation. We start by characterizing the amount and dynamics of the spare memory in Azure. We then design and implement memory-harvesting VMs (MHVMs), introducing new techniques for memory buffering, batching, and pre-reclamation. To demonstrate the use of MHVMs, we also extend a popular cluster scheduling framework (Hadoop) and a FaaS platform to adapt to them. Our main results show that (1) there is plenty of scope for memory harvesting in real platforms; (2) MHVMs are effective at mitigating the negative impacts of harvesting; and (3) our extensions of Hadoop and FaaS successfully hide the MHVMs’ varying memory size from the users’ data-processing jobs and functions. We conclude that memory harvesting has great potential for practical deployment and users can save up to 93% of their costs when running workloads on MHVMs. @InProceedings{ASPLOS22p583, author = {Alexander Fuerst and Stanko Novaković and Íñigo Goiri and Gohar Irfan Chaudhry and Prateek Sharma and Kapil Arya and Kevin Broas and Eugene Bak and Mehmet Iyigun and Ricardo Bianchini}, title = {Memory-Harvesting VMs in Cloud Platforms}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {583--594}, doi = {10.1145/3503222.3507725}, year = {2022}, } Publisher's Version
	Butt, Andrew	ASPLOS '22: "PLD: Fast FPGA Compilation ..." PLD: Fast FPGA Compilation to Make Reconfigurable Acceleration Compatible with Modern Incremental Refinement Software Development Yuanlong Xiao, Eric Micallef, Andrew Butt, Matthew Hofmann, Marc Alston, Matthew Goldsmith, Andrew Merczynski-Hait, and André DeHon (University of Pennsylvania, USA) FPGA-based accelerators are demonstrating significant absolute performance and energy efficiency compared with general-purpose CPUs. While FPGA computations can now be described in standard, programming languages, like C, development for FPGAs accelerators remains tedious and inaccessible to modern software engineers. Slow compiles (potentially taking tens of hours) inhibit the rapid, incremental refinement of designs that is the hallmark of modern software engineering. To address this issue, we introduce separate compilation and linkage into the FPGA design flow, providing faster design turns more familiar to software development. To realize this flow, we provide abstractions, compiler options, and compiler flow that allow the same C source code to be compiled to processor cores in seconds and to FPGA regions in minutes, providing the missing -O0 and -O1 options familiar in software development. This raises the FPGA programming level and standardizes the programming experience, bringing FPGA-based accelerators into a more familiar software platform ecosystem for software engineers. @InProceedings{ASPLOS22p933, author = {Yuanlong Xiao and Eric Micallef and Andrew Butt and Matthew Hofmann and Marc Alston and Matthew Goldsmith and Andrew Merczynski-Hait and André DeHon}, title = {PLD: Fast FPGA Compilation to Make Reconfigurable Acceleration Compatible with Modern Incremental Refinement Software Development}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {933--945}, doi = {10.1145/3503222.3507740}, year = {2022}, } Publisher's Version
	Byun, Ilkwon	ASPLOS '22: "CryoWire: Wire-Driven Microarchitecture ..." CryoWire: Wire-Driven Microarchitecture Designs for Cryogenic Computing Dongmoon Min, Yujin Chung, Ilkwon Byun, Junpyo Kim, and Jangwoo Kim (Seoul National University, South Korea) Cryogenic computing, which runs a computer device at an extremely low temperature, is promising thanks to its significant reduction of wire resistance as well as leakage current. Recent studies on cryogenic computing have focused on various architectural units including the main memory, cache, and CPU core running at 77K. However, little research has been conducted to fully exploit the fast cryogenic wires, even though the slow wires are becoming more serious performance bottleneck in modern processors. In this paper, we propose a CPU microarchitecture which extensively exploits the fast wires at 77K. For this goal, we first introduce our validated cryogenic-performance models for the CPU pipeline and network on chip (NoC), whose performance can be significantly limited by the slow wires. Next, based on the analysis with the models, we architect CryoSP and CryoBus as our pipeline and NoC designs to fully exploit the fast wires. Our evaluation shows that our cryogenic computer equipped with both microarchitectures achieves 3.82 times higher system-level performance compared to the conventional computer system thanks to the 96% higher clock frequency of CryoSP and five times lower NoC latency of CryoBus. @InProceedings{ASPLOS22p903, author = {Dongmoon Min and Yujin Chung and Ilkwon Byun and Junpyo Kim and Jangwoo Kim}, title = {CryoWire: Wire-Driven Microarchitecture Designs for Cryogenic Computing}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {903--917}, doi = {10.1145/3503222.3507749}, year = {2022}, } Publisher's Version
	Campanoni, Simone	ASPLOS '22: "CARAT CAKE: Replacing Paging ..." CARAT CAKE: Replacing Paging via Compiler/Kernel Cooperation Brian Suchy, Souradip Ghosh, Drew Kersnar, Siyuan Chai, Zhen Huang, Aaron Nelson, Michael Cuevas, Alex Bernat, Gaurav Chaudhary, Nikos Hardavellas, Simone Campanoni, and Peter Dinda (Northwestern University, USA) Virtual memory, specifically paging, is undergoing significant innovation due to being challenged by new demands from modern workloads. Recent work has demonstrated an alternative software only design that can result in simplified hardware requirements, even supporting purely physical addressing. While we have made the case for this Compiler- And Runtime-based Address Translation (CARAT) concept, its evaluation was based on a user-level prototype. We now report on incorporating CARAT into a kernel, forming Compiler- And Runtime-based Address Translation for CollAborative Kernel Environments (CARAT CAKE). In our implementation, a Linux-compatible x64 process abstraction can be based either on CARAT CAKE, or on a sophisticated paging implementation. Implementing CARAT CAKE involves kernel changes and compiler optimizations/transformations that must work on all code in the system, including kernel code. We evaluate CARAT CAKE in comparison with paging and find that CARAT CAKE is able to achieve the functionality of paging (protection, mapping, and movement properties) with minimal overhead. In turn, CARAT CAKE allows significant new benefits for systems including energy savings, larger L1 caches, and arbitrary granularity memory management. @InProceedings{ASPLOS22p98, author = {Brian Suchy and Souradip Ghosh and Drew Kersnar and Siyuan Chai and Zhen Huang and Aaron Nelson and Michael Cuevas and Alex Bernat and Gaurav Chaudhary and Nikos Hardavellas and Simone Campanoni and Peter Dinda}, title = {CARAT CAKE: Replacing Paging via Compiler/Kernel Cooperation}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {98--114}, doi = {10.1145/3503222.3507771}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available
	Chai, Siyuan	ASPLOS '22: "CARAT CAKE: Replacing Paging ..." CARAT CAKE: Replacing Paging via Compiler/Kernel Cooperation Brian Suchy, Souradip Ghosh, Drew Kersnar, Siyuan Chai, Zhen Huang, Aaron Nelson, Michael Cuevas, Alex Bernat, Gaurav Chaudhary, Nikos Hardavellas, Simone Campanoni, and Peter Dinda (Northwestern University, USA) Virtual memory, specifically paging, is undergoing significant innovation due to being challenged by new demands from modern workloads. Recent work has demonstrated an alternative software only design that can result in simplified hardware requirements, even supporting purely physical addressing. While we have made the case for this Compiler- And Runtime-based Address Translation (CARAT) concept, its evaluation was based on a user-level prototype. We now report on incorporating CARAT into a kernel, forming Compiler- And Runtime-based Address Translation for CollAborative Kernel Environments (CARAT CAKE). In our implementation, a Linux-compatible x64 process abstraction can be based either on CARAT CAKE, or on a sophisticated paging implementation. Implementing CARAT CAKE involves kernel changes and compiler optimizations/transformations that must work on all code in the system, including kernel code. We evaluate CARAT CAKE in comparison with paging and find that CARAT CAKE is able to achieve the functionality of paging (protection, mapping, and movement properties) with minimal overhead. In turn, CARAT CAKE allows significant new benefits for systems including energy savings, larger L1 caches, and arbitrary granularity memory management. @InProceedings{ASPLOS22p98, author = {Brian Suchy and Souradip Ghosh and Drew Kersnar and Siyuan Chai and Zhen Huang and Aaron Nelson and Michael Cuevas and Alex Bernat and Gaurav Chaudhary and Nikos Hardavellas and Simone Campanoni and Peter Dinda}, title = {CARAT CAKE: Replacing Paging via Compiler/Kernel Cooperation}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {98--114}, doi = {10.1145/3503222.3507771}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available
	Chaudhary, Gaurav	ASPLOS '22: "CARAT CAKE: Replacing Paging ..." CARAT CAKE: Replacing Paging via Compiler/Kernel Cooperation Brian Suchy, Souradip Ghosh, Drew Kersnar, Siyuan Chai, Zhen Huang, Aaron Nelson, Michael Cuevas, Alex Bernat, Gaurav Chaudhary, Nikos Hardavellas, Simone Campanoni, and Peter Dinda (Northwestern University, USA) Virtual memory, specifically paging, is undergoing significant innovation due to being challenged by new demands from modern workloads. Recent work has demonstrated an alternative software only design that can result in simplified hardware requirements, even supporting purely physical addressing. While we have made the case for this Compiler- And Runtime-based Address Translation (CARAT) concept, its evaluation was based on a user-level prototype. We now report on incorporating CARAT into a kernel, forming Compiler- And Runtime-based Address Translation for CollAborative Kernel Environments (CARAT CAKE). In our implementation, a Linux-compatible x64 process abstraction can be based either on CARAT CAKE, or on a sophisticated paging implementation. Implementing CARAT CAKE involves kernel changes and compiler optimizations/transformations that must work on all code in the system, including kernel code. We evaluate CARAT CAKE in comparison with paging and find that CARAT CAKE is able to achieve the functionality of paging (protection, mapping, and movement properties) with minimal overhead. In turn, CARAT CAKE allows significant new benefits for systems including energy savings, larger L1 caches, and arbitrary granularity memory management. @InProceedings{ASPLOS22p98, author = {Brian Suchy and Souradip Ghosh and Drew Kersnar and Siyuan Chai and Zhen Huang and Aaron Nelson and Michael Cuevas and Alex Bernat and Gaurav Chaudhary and Nikos Hardavellas and Simone Campanoni and Peter Dinda}, title = {CARAT CAKE: Replacing Paging via Compiler/Kernel Cooperation}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {98--114}, doi = {10.1145/3503222.3507771}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available
	Chaudhry, Gohar Irfan	ASPLOS '22: "Memory-Harvesting VMs in Cloud ..." Memory-Harvesting VMs in Cloud Platforms Alexander Fuerst, Stanko Novaković, Íñigo Goiri, Gohar Irfan Chaudhry, Prateek Sharma, Kapil Arya, Kevin Broas, Eugene Bak, Mehmet Iyigun, and Ricardo Bianchini (Indiana University, USA; Microsoft Research, USA; Microsoft Azure, USA) loud platforms monetize their spare capacity by renting “Spot” virtual machines (VMs) that can be evicted in favor of higher-priority VMs. Recent work has shown that resource-harvesting VMs are more effective at exploiting spare capacity than Spot VMs, while also reducing the number of evictions. However, the prior work focused on harvesting CPU cores while keeping memory size fixed. This wastes a substantial monetization opportunity and may even limit the ability of harvesting VMs to leverage spare cores. Thus, in this paper, we explore memory harvesting and its challenges in real cloud platforms, namely its impact on VM creation time, NUMA spanning, and page fragmentation. We start by characterizing the amount and dynamics of the spare memory in Azure. We then design and implement memory-harvesting VMs (MHVMs), introducing new techniques for memory buffering, batching, and pre-reclamation. To demonstrate the use of MHVMs, we also extend a popular cluster scheduling framework (Hadoop) and a FaaS platform to adapt to them. Our main results show that (1) there is plenty of scope for memory harvesting in real platforms; (2) MHVMs are effective at mitigating the negative impacts of harvesting; and (3) our extensions of Hadoop and FaaS successfully hide the MHVMs’ varying memory size from the users’ data-processing jobs and functions. We conclude that memory harvesting has great potential for practical deployment and users can save up to 93% of their costs when running workloads on MHVMs. @InProceedings{ASPLOS22p583, author = {Alexander Fuerst and Stanko Novaković and Íñigo Goiri and Gohar Irfan Chaudhry and Prateek Sharma and Kapil Arya and Kevin Broas and Eugene Bak and Mehmet Iyigun and Ricardo Bianchini}, title = {Memory-Harvesting VMs in Cloud Platforms}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {583--594}, doi = {10.1145/3503222.3507725}, year = {2022}, } Publisher's Version
	Chen, Gang	ASPLOS '22: "NVAlloc: Rethinking Heap Metadata ..." NVAlloc: Rethinking Heap Metadata Management in Persistent Memory Allocators Zheng Dang, Shuibing He, Peiyi Hong, Zhenxin Li, Xuechen Zhang, Xian-He Sun, and Gang Chen (Zhejiang University, China; Washington State University, USA; Illinois Institute of Technology, USA) Persistent memory allocation is a fundamental building block for developing high-performance and in-memory applications. Existing persistent memory allocators suffer from suboptimal heap organizations that introduce repeated cache line flushes and small random accesses in persistent memory. Worse, many allocators use static slab segregation resulting in a dramatic increase in memory consumption when allocation request size is changed. In this paper, we design a novel allocator, named NVAlloc, to solve the above issues simultaneously. First, NVAlloc eliminates cache line reflushes by mapping contiguous data blocks in slabs to interleaved metadata entries stored in different cache lines. Second, it writes small metadata units to a persistent bookkeeping log in a sequential pattern to remove random heap metadata accesses in persistent memory. Third, instead of using static slab segregation, it supports slab morphing, which allows slabs to be transformed between size classes to significantly improve slab usage. NVAlloc is complementary to the existing consistency models. Results on 6 benchmarks demonstrate that NVAlloc improves the performance of state-of-the-art persistent memory allocators by up to 6.4x and 57x for small and large allocations, respectively. Using NVAlloc reduces memory usage by up to 57.8%. Besides, we integrate NVAlloc in a persistent FPTree. Compared to the state-of-the-art allocators, NVAlloc improves the performance of this application by up to 3.1x. @InProceedings{ASPLOS22p115, author = {Zheng Dang and Shuibing He and Peiyi Hong and Zhenxin Li and Xuechen Zhang and Xian-He Sun and Gang Chen}, title = {NVAlloc: Rethinking Heap Metadata Management in Persistent Memory Allocators}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {115--127}, doi = {10.1145/3503222.3507743}, year = {2022}, } Publisher's Version
	Chen, Haibo	ASPLOS '22: "Serverless Computing on Heterogeneous ..." Serverless Computing on Heterogeneous Computers Dong Du, Qingyuan Liu, Xueqiang Jiang, Yubin Xia, Binyu Zang, and Haibo Chen (Shanghai Jiao Tong University, China; Shanghai Artificial Intelligence Laboratory, China) Existing serverless computing platforms are built upon homogeneous computers, limiting the function density and restricting serverless computing to limited scenarios. We introduce Molecule, the first serverless computing system utilizing heterogeneous computers. Molecule enables both general-purpose devices (e.g., Nvidia DPU) and domain-specific accelerators (e.g., FPGA and GPU) for serverless applications that significantly improve function density (50% higher) and application performance (up to 34.6x). To achieve these results, we first propose XPU-Shim, a distributed shim to bridge the gap between underlying multi-OS systems (when using general-purpose devices) and our serverless runtime (i.e., Molecule). We further introduce vectorized sandbox, a sandbox abstraction to abstract hardware heterogeneity (when using domain-specific accelerators). Moreover, we also review state-of-the-art serverless optimizations on startup and communication latency and overcome the challenges to implement them on heterogeneous computers. We have implemented Molecule on real platforms with Nvidia DPUs and Xilinx FPGAs and evaluate it using benchmarks and real-world applications. @InProceedings{ASPLOS22p797, author = {Dong Du and Qingyuan Liu and Xueqiang Jiang and Yubin Xia and Binyu Zang and Haibo Chen}, title = {Serverless Computing on Heterogeneous Computers}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {797--813}, doi = {10.1145/3503222.3507732}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Chen, Jingji	ASPLOS '22: "SparseCore: Stream ISA and ..." SparseCore: Stream ISA and Processor Specialization for Sparse Computation Gengyu Rao, Jingji Chen, Jason Yik, and Xuehai Qian (University of Southern California, USA) Computation on sparse data is becoming increasingly important for many applications. Recent sparse computation accelerators are designed for specific algorithm/application, making them inflexible with software optimizations. This paper proposes SparseCore, the first general-purpose processor extension for sparse computation that can flexibly accelerate complex code patterns and fast-evolving algorithms. We extend the instruction set architecture (ISA) to make stream or sparse vector first-class citizens, and develop efficient architectural components to support the stream ISA. The novel ISA extension intrinsically operates on streams, realizing both efficient data movement and computation. The simulation results show that SparseCore achieves significant speedups for sparse tensor computation and graph pattern computation. @InProceedings{ASPLOS22p186, author = {Gengyu Rao and Jingji Chen and Jason Yik and Xuehai Qian}, title = {SparseCore: Stream ISA and Processor Specialization for Sparse Computation}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {186--199}, doi = {10.1145/3503222.3507705}, year = {2022}, } Publisher's Version
	Chen, Li	ASPLOS '22: "NASPipe: High Performance ..." NASPipe: High Performance and Reproducible Pipeline Parallel Supernet Training via Causal Synchronous Parallelism Shixiong Zhao, Fanxin Li, Xusheng Chen, Tianxiang Shen, Li Chen, Sen Wang, Nicholas Zhang, Cheng Li, and Heming Cui (University of Hong Kong, China; Huawei Technologies, China; University of Science and Technology of China, China) Supernet training, a prevalent and important paradigm in Neural Architecture Search, embeds the whole DNN architecture search space into one monolithic supernet, iteratively activates a subset of the supernet (i.e., a subnet) for fitting each batch of data, and searches a high-quality subnet which meets specific requirements. Although training subnets in parallel on multiple GPUs is desirable for acceleration, there inherently exists a race hazard that concurrent subnets may access the same DNN layers. Existing systems support neither efficiently parallelizing subnets’ training executions, nor resolving the race hazard deterministically, leading to unreproducible training procedures and potentiallly non-trivial accuracy loss. We present NASPipe, the first high-performance and reproducible distributed supernet training system via causal synchronous parallel (CSP) pipeline scheduling abstraction: NASPipe partitions a supernet across GPUs and concurrently executes multiple generated sub-tasks (subnets) in a pipelined manner; meanwhile, it oversees the correlations between the subnets and deterministically resolves any causal dependency caused by subnets’ layer sharing. To obtain high performance, NASPipe’s CSP scheduler exploits the fact that the larger a supernet spans, the fewer dependencies manifest between chronologically close subnets; therefore, it aggressively schedules the subnets with larger chronological orders into execution, only if they are not causally dependent on unfinished precedent subnets. Moreover, to relieve the excessive GPU memory burden for holding the whole supernet’s parameters, NASPipe uses a context switch technique that stashes the whole supernet in CPU memory, precisely predicts the subnets’ schedule, and pre-fetches/evicts a subnet before/after its execution. The evaluation shows that NASPipe is the only system that retains supernet training reproducibility, while achieving a comparable and even higher performance (up to 7.8X) compared to three recent pipeline training systems (e.g., GPipe). @InProceedings{ASPLOS22p374, author = {Shixiong Zhao and Fanxin Li and Xusheng Chen and Tianxiang Shen and Li Chen and Sen Wang and Nicholas Zhang and Cheng Li and Heming Cui}, title = {NASPipe: High Performance and Reproducible Pipeline Parallel Supernet Training via Causal Synchronous Parallelism}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {374--387}, doi = {10.1145/3503222.3507735}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Chen, Qihang	ASPLOS '22: "FINGERS: Exploiting Fine-Grained ..." FINGERS: Exploiting Fine-Grained Parallelism in Graph Mining Accelerators Qihang Chen, Boyu Tian, and Mingyu Gao (Tsinghua University, China) Graph mining is an emerging application of high importance and also with high complexity, thus requiring efficient hardware acceleration. Current accelerator designs only utilize coarse-grained parallelism, leaving large room for further optimizations. Our key insight is to fully exploit fine-grained parallelism to overcome the existing issues of hardware underutilization, inefficient resource provision, and limited single-thread performance under imbalanced loads. Targeting pattern-aware graph mining algorithms, we first comprehensively identify and analyze the abundant fine-grained parallelism at the branch, set, and segment levels during search tree exploration and set operations. We then propose a novel graph mining accelerator, FINGERS, which effectively exploits these multiple levels of fine-grained parallelism to achieve significant performance improvements. FINGERS mainly enhances the design of each single processing element with parallel compute units for set operations, and efficient techniques for task scheduling, load balancing, and data aggregation. FINGERS outperforms the state-of-the-art design by 2.8× on average and up to 8.9× with the same chip area. We also demonstrate that different patterns and different graphs exhibit drastically different parallelism opportunities, justifying the necessity of exploiting all levels of fine-grained parallelism in FINGERS. @InProceedings{ASPLOS22p43, author = {Qihang Chen and Boyu Tian and Mingyu Gao}, title = {FINGERS: Exploiting Fine-Grained Parallelism in Graph Mining Accelerators}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {43--55}, doi = {10.1145/3503222.3507730}, year = {2022}, } Publisher's Version
	Chen, Quan	ASPLOS '22: "VELTAIR: Towards High-Performance ..." VELTAIR: Towards High-Performance Multi-tenant Deep Learning Services via Adaptive Compilation and Scheduling Zihan Liu, Jingwen Leng, Zhihui Zhang, Quan Chen, Chao Li, and Minyi Guo (Shanghai Jiao Tong University, China; Shanghai Qi Zhi Institute, China) Deep learning (DL) models have achieved great success in many application domains. As such, many industrial companies such as Google and Facebook have acknowledged the importance of multi-tenant DL services. Although the multi-tenant service has been studied in conventional workloads, it is not been deeply studied on deep learning service, especially on general-purpose hardware. In this work, we systematically analyze the opportunities and challenges of providing multi-tenant deep learning services on the general-purpose CPU architecture from the aspects of scheduling granularity and code generation. We propose an adaptive granularity scheduling scheme to both guarantee resource usage efficiency and reduce the scheduling conflict rate. We also propose an adaptive compilation strategy, by which we can dynamically and intelligently pick a program with proper exclusive and shared resource usage to reduce overall interference-induced performance loss. Compared to the existing works, our design can serve more requests under the same QoS target in various scenarios (e.g., +71%, +62%, +45% for light, medium, and heavy workloads, respectively), and reduce the averaged query latency by 50%. @InProceedings{ASPLOS22p388, author = {Zihan Liu and Jingwen Leng and Zhihui Zhang and Quan Chen and Chao Li and Minyi Guo}, title = {VELTAIR: Towards High-Performance Multi-tenant Deep Learning Services via Adaptive Compilation and Scheduling}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {388--401}, doi = {10.1145/3503222.3507752}, year = {2022}, } Publisher's Version ASPLOS '22: "Astraea: Towards QoS-Aware ..." Astraea: Towards QoS-Aware and Resource-Efficient Multi-stage GPU Services Wei Zhang, Quan Chen, Kaihua Fu, Ningxin Zheng, Zhiyi Huang, Jingwen Leng, and Minyi Guo (Shanghai Jiao Tong University, China; Microsoft Research, China; University of Otago, New Zealand) Multi-stage user-facing applications on GPUs are widely-used nowa- days, and are often implemented to be microservices. Prior re- search works are not applicable to ensuring QoS of GPU-based microservices due to the different communication patterns and shared resource contentions. We propose Astraea to manage GPU microservices considering the above factors. In Astraea, a microser- vice deployment policy is used to maximize the supported peak service load while ensuring the required QoS. To adaptively switch the communication methods between microservices according to different deployments, we propose an auto-scaling GPU communi- cation framework. The framework automatically scales based on the currently used hardware topology and microservice location, and adopts global memory-based techniques to reduce intra-GPU communication. Astraea increases the supported peak load by up to 82.3% while achieving the desired 99%-ile latency target compared with state-of-the-art solutions. @InProceedings{ASPLOS22p570, author = {Wei Zhang and Quan Chen and Kaihua Fu and Ningxin Zheng and Zhiyi Huang and Jingwen Leng and Minyi Guo}, title = {Astraea: Towards QoS-Aware and Resource-Efficient Multi-stage GPU Services}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {570--582}, doi = {10.1145/3503222.3507721}, year = {2022}, } Publisher's Version ASPLOS '22: "FaaSFlow: Enable Efficient ..." FaaSFlow: Enable Efficient Workflow Execution for Function-as-a-Service Zijun Li, Yushi Liu, Linsong Guo, Quan Chen, Jiagan Cheng, Wenli Zheng, and Minyi Guo (Shanghai Jiao Tong University, China) Serverless computing (Function-as-a-Service) provides fine-grain resource sharing by running functions (or Lambdas) in containers. Data-dependent functions are required to be invoked following a pre-defined logic, which is known as serverless workflows. However, our investigation shows that the traditional master-worker based workflow execution architecture performs poorly in serverless context. One significant overhead results from the master-side workflow schedule pattern, with which the functions are triggered in the master node and assigned to worker nodes for execution. Besides, the data movement between workers also reduces the throughput. To this end, we present a worker-side workflow schedule pattern for serverless workflow execution. Following the design, we implement FaaSFlow to enable efficient workflow execution in the serverless context. Besides, we propose an adaptive storage library FaaStore that enables fast data transfer between functions on the same node without through the database. Experiment results show that FaaSFlow effectively mitigates the workflow scheduling overhead by 74.6% on average and data transmission overhead by 95% at most. When the network bandwidth fluctuates, FaaSFlow-FaaStore reduces the throughput degradation by 23.0%, and is able to multiply the utilization of network bandwidth by 1.5X-4X. @InProceedings{ASPLOS22p782, author = {Zijun Li and Yushi Liu and Linsong Guo and Quan Chen and Jiagan Cheng and Wenli Zheng and Minyi Guo}, title = {FaaSFlow: Enable Efficient Workflow Execution for Function-as-a-Service}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {782--796}, doi = {10.1145/3503222.3507717}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Chen, Ruirong	ASPLOS '22: "Eavesdropping User Credentials ..." Eavesdropping User Credentials via GPU Side Channels on Smartphones Boyuan Yang, Ruirong Chen, Kai Huang, Jun Yang, and Wei Gao (University of Pittsburgh, USA) Graphics Processing Unit (GPU) on smartphones is an effective target for hardware attacks. In this paper, we present a new side channel attack on mobile GPUs of Android smartphones, allowing an unprivileged attacker to eavesdrop the user's credentials, such as login usernames and passwords, from their inputs through on-screen keyboard. Our attack targets on Qualcomm Adreno GPUs and investigate the amount of GPU overdraw when rendering the popups of user's key presses of inputs. Such GPU overdraw caused by each key press corresponds to unique variations of selected GPU performance counters, from which these key presses can be accurately inferred. Experiment results from practical use on multiple models of Android smartphones show that our attack can correctly infer more than 80% of user's credential inputs, but incur negligible amounts of computing overhead and network traffic on the victim device. To counter this attack, this paper suggests mitigations of access control on GPU performance counters, or applying obfuscations on the values of GPU performance counters. @InProceedings{ASPLOS22p285, author = {Boyuan Yang and Ruirong Chen and Kai Huang and Jun Yang and Wei Gao}, title = {Eavesdropping User Credentials via GPU Side Channels on Smartphones}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {285--299}, doi = {10.1145/3503222.3507757}, year = {2022}, } Publisher's Version Published Artifact Video Artifacts Available Artifacts Functional
	Chen, Shutong	ASPLOS '22: "CoolEdge: Hotspot-Relievable ..." CoolEdge: Hotspot-Relievable Warm Water Cooling for Energy-Efficient Edge Datacenters Qiangyu Pei, Shutong Chen, Qixia Zhang, Xinhui Zhu, Fangming Liu, Ziyang Jia, Yishuo Wang, and Yongjie Yuan (Huazhong University of Science and Technology, China) As the computing frontier drifts to the edge, edge datacenters play a crucial role in supporting various real-time applications. Different from cloud datacenters, the requirements of proximity to end-users, high density, and heterogeneity, present new challenges to cool the edge datacenters efficiently. Although warm water cooling has become a promising cooling technique for this infrastructure, the one-size-fits-all cooling control would lower the cooling efficiency considerably because of the severe thermal imbalance across servers, hardware, and even inside one hardware component in an edge datacenter. In this work, we propose CoolEdge, a hotspot-relievable warm water cooling system for improving the cooling efficiency and saving costs of edge datacenters. Specifically, through the elaborate design of water circulations, CoolEdge can dynamically adjust the water temperature and flow rate for each heterogeneous hardware component to eliminate the hardware-level hotspots. By redesigning cold plates, CoolEdge can quickly disperse the chip-level hotspots without manual intervention. We further quantify the power saving achieved by the warm water cooling theoretically, and propose a custom-designed cooling solution to decide an appropriate water temperature and flow rate periodically. Based on a hardware prototype and real-world traces from SURFsara, the evaluation results show that CoolEdge reduces the cooling energy by 81.81% and 71.92%, respectively, compared with conventional and state-of-the-art water cooling systems. @InProceedings{ASPLOS22p814, author = {Qiangyu Pei and Shutong Chen and Qixia Zhang and Xinhui Zhu and Fangming Liu and Ziyang Jia and Yishuo Wang and Yongjie Yuan}, title = {CoolEdge: Hotspot-Relievable Warm Water Cooling for Energy-Efficient Edge Datacenters}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {814--829}, doi = {10.1145/3503222.3507713}, year = {2022}, } Publisher's Version
	Chen, Xingzhen	ASPLOS '22: "INFless: A Native Serverless ..." INFless: A Native Serverless System for Low-Latency, High-Throughput Inference Yanan Yang, Laiping Zhao, Yiming Li, Huanyu Zhang, Jie Li, Mingyang Zhao, Xingzhen Chen, and Keqiu Li (Tianjin University, China; 58.com, China) Modern websites increasingly rely on machine learning (ML) to improve their business efficiency. Developing and maintaining ML services incurs high costs for developers. Although serverless systems are a promising solution to reduce costs, we find that the current general purpose serverless systems cannot meet the low latency, high throughput demands of ML services. While simply ”patching” general serverless systems does not resolve the problem completely, we propose that such a system should natively combine the features of inference with a serverless paradigm. We present INFless, the first ML domain-specific serverless platform. It provides a unified, heterogeneous resource abstraction between CPU and accelerators, and achieves high throughput using built-in batching and non-uniform scaling mechanisms. It also supports low latency through coordinated management of batch queuing time, execution time and coldstart rate. We evaluate INFless using both a local cluster testbed and a large-scale simulation. Experimental results show that INFless outperforms state-of-the-art systems by 2×-5× on system throughput, meeting the latency goals of ML services. @InProceedings{ASPLOS22p768, author = {Yanan Yang and Laiping Zhao and Yiming Li and Huanyu Zhang and Jie Li and Mingyang Zhao and Xingzhen Chen and Keqiu Li}, title = {INFless: A Native Serverless System for Low-Latency, High-Throughput Inference}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {768--781}, doi = {10.1145/3503222.3507709}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available
	Chen, Xusheng	ASPLOS '22: "NASPipe: High Performance ..." NASPipe: High Performance and Reproducible Pipeline Parallel Supernet Training via Causal Synchronous Parallelism Shixiong Zhao, Fanxin Li, Xusheng Chen, Tianxiang Shen, Li Chen, Sen Wang, Nicholas Zhang, Cheng Li, and Heming Cui (University of Hong Kong, China; Huawei Technologies, China; University of Science and Technology of China, China) Supernet training, a prevalent and important paradigm in Neural Architecture Search, embeds the whole DNN architecture search space into one monolithic supernet, iteratively activates a subset of the supernet (i.e., a subnet) for fitting each batch of data, and searches a high-quality subnet which meets specific requirements. Although training subnets in parallel on multiple GPUs is desirable for acceleration, there inherently exists a race hazard that concurrent subnets may access the same DNN layers. Existing systems support neither efficiently parallelizing subnets’ training executions, nor resolving the race hazard deterministically, leading to unreproducible training procedures and potentiallly non-trivial accuracy loss. We present NASPipe, the first high-performance and reproducible distributed supernet training system via causal synchronous parallel (CSP) pipeline scheduling abstraction: NASPipe partitions a supernet across GPUs and concurrently executes multiple generated sub-tasks (subnets) in a pipelined manner; meanwhile, it oversees the correlations between the subnets and deterministically resolves any causal dependency caused by subnets’ layer sharing. To obtain high performance, NASPipe’s CSP scheduler exploits the fact that the larger a supernet spans, the fewer dependencies manifest between chronologically close subnets; therefore, it aggressively schedules the subnets with larger chronological orders into execution, only if they are not causally dependent on unfinished precedent subnets. Moreover, to relieve the excessive GPU memory burden for holding the whole supernet’s parameters, NASPipe uses a context switch technique that stashes the whole supernet in CPU memory, precisely predicts the subnets’ schedule, and pre-fetches/evicts a subnet before/after its execution. The evaluation shows that NASPipe is the only system that retains supernet training reproducibility, while achieving a comparable and even higher performance (up to 7.8X) compared to three recent pipeline training systems (e.g., GPipe). @InProceedings{ASPLOS22p374, author = {Shixiong Zhao and Fanxin Li and Xusheng Chen and Tianxiang Shen and Li Chen and Sen Wang and Nicholas Zhang and Cheng Li and Heming Cui}, title = {NASPipe: High Performance and Reproducible Pipeline Parallel Supernet Training via Causal Synchronous Parallelism}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {374--387}, doi = {10.1145/3503222.3507735}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Chen, Yanju	ASPLOS '22: "Tree Traversal Synthesis Using ..." Tree Traversal Synthesis Using Domain-Specific Symbolic Compilation Yanju Chen, Junrui Liu, Yu Feng, and Rastislav Bodik (University of California at Santa Barbara, USA; University of Washington, USA) Efficient computation on tree data structures is important in compilers, numeric computations, and web browser layout engines. Efficiency is achieved by statically scheduling the computation into a small number of tree traversals and by performing the traversals in parallel when possible. Manual design of such traversals leads to bugs, as observed in web browsers. Automatic schedulers avoid these bugs but they currently cannot explore a space of legal traversals, which prevents exploring the trade-offs between parallelism and minimizing the number of traversals. We describe Hecate, a synthesizer of tree traversals that can produce both serial and parallel traversals. A key feature is that the synthesizer is extensible by the programmer who can define a template for new kinds of traversals. Hecate is constructed as a solver-aided domain-specific language, meaning that the synthesizer is generated automatically by translating the tree traversal DSL to an SMT solver that synthesizes the traversals. We improve on the general-purpose solver-aided architecture with a scheduling-specific symbolic evaluation that maintains the engineering advantages solver-aided design but generates efficient ILP encoding that is much more efficient to solve than SMT constraints. On the set of Grafter problems, Hecate synthesizes traversals that trade off traversal fusion to exploit parallelism. Additionally, Hecate allows defining a tree data structure with an arbitrary number of children. Together, parallelism and data structure improvements accelerate the computation 2× on a tree rendering problem. Finally, Hecate’s domain-specific symbolic compilation accelerates synthesis 3× compared to the general-purpose compilation to an SMT solver; when scheduling a CSS engine traversal, this ILP-based synthesis executes orders of magnitude faster. @InProceedings{ASPLOS22p1030, author = {Yanju Chen and Junrui Liu and Yu Feng and Rastislav Bodik}, title = {Tree Traversal Synthesis Using Domain-Specific Symbolic Compilation}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1030--1042}, doi = {10.1145/3503222.3507751}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional
	Chen, Zhangyu	ASPLOS '22: "Efficiently Detecting Concurrency ..." Efficiently Detecting Concurrency Bugs in Persistent Memory Programs Zhangyu Chen, Yu Hua, Yongle Zhang, and Luochangqi Ding (Huazhong University of Science and Technology, China; Purdue University, USA) Due to the salient DRAM-comparable performance, TB-scale capacity, and non-volatility, persistent memory (PM) provides new opportunities for large-scale in-memory computing with instant crash recovery. However, programming PM systems is error-prone due to the existence of crash-consistency bugs, which are challenging to diagnose especially with concurrent programming widely adopted in PM applications to exploit hardware parallelism. Existing bug detection tools for DRAM-based concurrency issues cannot detect PM crash-consistency bugs because they are oblivious to PM operations and PM consistency. On the other hand, existing PM-specific debugging tools only focus on sequential PM programs and cannot effectively detect crash-consistency issues hidden in concurrent executions. In order to effectively detect crash-consistency bugs that only manifest in concurrent executions, we propose PMRace, the first PM-specific concurrency bug detection tool. We identify and define two new types of concurrent crash-consistency bugs: PM Inter-thread Inconsistency and PM Synchronization Inconsistency. In particular, PMRace adopts PM-aware and coverage-guided fuzz testing to explore PM program executions. For PM Inter-thread Inconsistency, which denotes the data inconsistency hidden in thread interleavings, PMRace performs PM-aware interleaving exploration and thread scheduling to drive the execution towards executions that reveal such inconsistencies. For PM Synchronization Inconsistency between persisted synchronization variables and program data, PMRace identifies the inconsistency during interleaving exploration. The post-failure validation reduces the false positives that come from custom crash recovery mechanisms. PMRace has found 14 bugs (10 new bugs) in real-world concurrent PM systems including PM-version memcached. @InProceedings{ASPLOS22p873, author = {Zhangyu Chen and Yu Hua and Yongle Zhang and Luochangqi Ding}, title = {Efficiently Detecting Concurrency Bugs in Persistent Memory Programs}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {873--887}, doi = {10.1145/3503222.3507755}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Chen, Zhaodong	ASPLOS '22: "DOTA: Detect and Omit Weak ..." DOTA: Detect and Omit Weak Attentions for Scalable Transformer Acceleration Zheng Qu, Liu Liu, Fengbin Tu, Zhaodong Chen, Yufei Ding, and Yuan Xie (University of California at Santa Barbara, USA) Transformer Neural Networks have demonstrated leading performance in many applications spanning over language understanding, image processing, and generative modeling. Despite the impressive performance, long-sequence Transformer processing is expensive due to quadratic computation complexity and memory consumption of self-attention. In this paper, we present DOTA, an algorithm-architecture co-design that effectively addresses the challenges of scalable Transformer inference. Based on the insight that not all connections in an attention graph are equally important, we propose to jointly optimize a lightweight Detector with the Transformer model to accurately detect and omit weak connections during runtime. Furthermore, we design a specialized system architecture for end-to-end Transformer acceleration using the proposed attention detection mechanism. Experiments on a wide range of benchmarks demonstrate the superior performance of DOTA over other solutions. In summary, DOTA achieves 152.6x and 4.5x performance speedup and orders of magnitude energy-efficiency improvements over GPU and customized hardware, respectively. @InProceedings{ASPLOS22p14, author = {Zheng Qu and Liu Liu and Fengbin Tu and Zhaodong Chen and Yufei Ding and Yuan Xie}, title = {DOTA: Detect and Omit Weak Attentions for Scalable Transformer Acceleration}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {14--26}, doi = {10.1145/3503222.3507738}, year = {2022}, } Publisher's Version
	Cheng, Jiagan	ASPLOS '22: "FaaSFlow: Enable Efficient ..." FaaSFlow: Enable Efficient Workflow Execution for Function-as-a-Service Zijun Li, Yushi Liu, Linsong Guo, Quan Chen, Jiagan Cheng, Wenli Zheng, and Minyi Guo (Shanghai Jiao Tong University, China) Serverless computing (Function-as-a-Service) provides fine-grain resource sharing by running functions (or Lambdas) in containers. Data-dependent functions are required to be invoked following a pre-defined logic, which is known as serverless workflows. However, our investigation shows that the traditional master-worker based workflow execution architecture performs poorly in serverless context. One significant overhead results from the master-side workflow schedule pattern, with which the functions are triggered in the master node and assigned to worker nodes for execution. Besides, the data movement between workers also reduces the throughput. To this end, we present a worker-side workflow schedule pattern for serverless workflow execution. Following the design, we implement FaaSFlow to enable efficient workflow execution in the serverless context. Besides, we propose an adaptive storage library FaaStore that enables fast data transfer between functions on the same node without through the database. Experiment results show that FaaSFlow effectively mitigates the workflow scheduling overhead by 74.6% on average and data transmission overhead by 95% at most. When the network bandwidth fluctuates, FaaSFlow-FaaStore reduces the throughput degradation by 23.0%, and is able to multiply the utilization of network bandwidth by 1.5X-4X. @InProceedings{ASPLOS22p782, author = {Zijun Li and Yushi Liu and Linsong Guo and Quan Chen and Jiagan Cheng and Wenli Zheng and Minyi Guo}, title = {FaaSFlow: Enable Efficient Workflow Execution for Function-as-a-Service}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {782--796}, doi = {10.1145/3503222.3507717}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Cheng, Xiang	ASPLOS '22: "Creating Concise and Efficient ..." Creating Concise and Efficient Dynamic Analyses with ALDA Xiang Cheng and David Devecsery (Georgia Institute of Technology, USA) Dynamic program analyses are essential to creating safe, reliable, and productive computing environments. However, these analyses are challenging and time-consuming to construct due to the low-level optimization required to achieve acceptable performance. Consequently, many analyses are often never realized, or have inefficient implementations. In this work we argue that many analyses can and should be constructed with a high-level description language, leaving the burden of low-level optimizations to the analysis instrumentation system itself. We propose a novel language for dynamic analysis called ALDA. ALDA leverages common structuring of dynamic analyses to provide a simple and high-level description for dynamic analyses. Through restricting the supported behaviors to only essential operations in dynamic analyses, an optimizing compiler for ALDA can create analysis implementations with performance on-par to hand-tuned analyses. To demonstrate ALDA’s universality and efficiency, we create an optimizing compiler for ALDA targeting the LLVM instrumentation framework named ALDAcc. We use ALDAcc to construct 8 different dynamic analysis algorithms, including the popular MemorySanitizer analysis, and show their construction is succinct and simple. By comparing two of them (Eraser and MemorySanitizer) with their hand-tuned implementations, we show that ALDAcc’s optimized analyses are comparable to hand-tuned implementations. @InProceedings{ASPLOS22p740, author = {Xiang Cheng and David Devecsery}, title = {Creating Concise and Efficient Dynamic Analyses with ALDA}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {740--752}, doi = {10.1145/3503222.3507760}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Cheung, Alvin	ASPLOS '22: "Vector Instruction Selection ..." Vector Instruction Selection for Digital Signal Processors using Program Synthesis Maaz Bin Safeer Ahmad, Alexander J. Root, Andrew Adams, Shoaib Kamil, and Alvin Cheung (Adobe, USA; Massachusetts Institute of Technology, USA; University of California at Berkeley, USA) Instruction selection, whereby input code represented in an intermediate representation is translated into executable instructions from the target platform, is often the most target-dependent component in optimizing compilers. Current approaches include pattern matching, which is brittle and tedious to design, or search-based methods, which are limited by scalability of the search algorithm. In this paper, we propose a new algorithm that first abstracts the target platform instructions into high-level uber-instructions, with each uber-instruction unifying multiple concrete instructions from the target platform. Program synthesis is used to lift input code sequences into semantically equivalent sequences of uber-instructions and then to lower from uber-instructions to machine code. Using 21 real-world benchmarks, we show that our synthesis-based instruction selection algorithm can generate instruction sequences for a hardware target, with the synthesized code performing up to 2.1x faster as compared to code generated by a professionally-developed optimizing compiler for the same platform. @InProceedings{ASPLOS22p1004, author = {Maaz Bin Safeer Ahmad and Alexander J. Root and Andrew Adams and Shoaib Kamil and Alvin Cheung}, title = {Vector Instruction Selection for Digital Signal Processors using Program Synthesis}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1004--1016}, doi = {10.1145/3503222.3507714}, year = {2022}, } Publisher's Version
	Cho, Haehyun	ASPLOS '22: "ViK: Practical Mitigation ..." ViK: Practical Mitigation of Temporal Memory Safety Violations through Object ID Inspection Haehyun Cho, Jinbum Park, Adam Oest, Tiffany Bao, Ruoyu Wang, Yan Shoshitaishvili, Adam Doupé, and Gail-Joon Ahn (Soongsil University, South Korea; Samsung Research, South Korea; PayPal, USA; Arizona State University, USA; Samsung Research, USA) Temporal memory safety violations, such as use-after-free (UAF) vulnerabilities, are a critical security issue for software written in memory-unsafe languages such as C and C++. In this paper, we introduce ViK, a novel, lightweight, and widely applicable runtime defense that can protect both operating system (OS) kernels and user-space applications against temporal memory safety violations. ViK performs object ID inspection, where it assigns a random identifier to every allocated object and stores the identifier in the unused bits of the corresponding pointer. When the pointer is used, ViK inspects the value of a pointer before dereferencing, ensuring that the pointer still references the original object. To the best of our knowledge, this is the first mitigation against temporal memory safety violations that scales to OS kernels. We evaluated the software prototype of ViK on Android and Linux kernels and observed runtime overhead of around 20%. Also, we evaluated a hardware-assisted prototype of ViK on Android kernel, where the runtime overhead was as low as 2%. @InProceedings{ASPLOS22p271, author = {Haehyun Cho and Jinbum Park and Adam Oest and Tiffany Bao and Ruoyu Wang and Yan Shoshitaishvili and Adam Doupé and Gail-Joon Ahn}, title = {ViK: Practical Mitigation of Temporal Memory Safety Violations through Object ID Inspection}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {271--284}, doi = {10.1145/3503222.3507780}, year = {2022}, } Publisher's Version Artifacts Functional
	Chung, Yujin	ASPLOS '22: "CryoWire: Wire-Driven Microarchitecture ..." CryoWire: Wire-Driven Microarchitecture Designs for Cryogenic Computing Dongmoon Min, Yujin Chung, Ilkwon Byun, Junpyo Kim, and Jangwoo Kim (Seoul National University, South Korea) Cryogenic computing, which runs a computer device at an extremely low temperature, is promising thanks to its significant reduction of wire resistance as well as leakage current. Recent studies on cryogenic computing have focused on various architectural units including the main memory, cache, and CPU core running at 77K. However, little research has been conducted to fully exploit the fast cryogenic wires, even though the slow wires are becoming more serious performance bottleneck in modern processors. In this paper, we propose a CPU microarchitecture which extensively exploits the fast wires at 77K. For this goal, we first introduce our validated cryogenic-performance models for the CPU pipeline and network on chip (NoC), whose performance can be significantly limited by the slow wires. Next, based on the analysis with the models, we architect CryoSP and CryoBus as our pipeline and NoC designs to fully exploit the fast wires. Our evaluation shows that our cryogenic computer equipped with both microarchitectures achieves 3.82 times higher system-level performance compared to the conventional computer system thanks to the 96% higher clock frequency of CryoSP and five times lower NoC latency of CryoBus. @InProceedings{ASPLOS22p903, author = {Dongmoon Min and Yujin Chung and Ilkwon Byun and Junpyo Kim and Jangwoo Kim}, title = {CryoWire: Wire-Driven Microarchitecture Designs for Cryogenic Computing}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {903--917}, doi = {10.1145/3503222.3507749}, year = {2022}, } Publisher's Version
	Cock, David	ASPLOS '22: "Enzian: An Open, General, ..." Enzian: An Open, General, CPU/FPGA Platform for Systems Software Research David Cock, Abishek Ramdas, Daniel Schwyn, Michael Giardino, Adam Turowski, Zhenhao He, Nora Hossle, Dario Korolija, Melissa Licciardello, Kristina Martsenko, Reto Achermann, Gustavo Alonso, and Timothy Roscoe (ETH Zurich, Switzerland; University of British Columbia, Canada) Hybrid computing platforms, comprising CPU cores and FPGA logic, are increasingly used for accelerating data-intensive workloads in cloud deployments, and are a growing topic of interest in systems research. However, from a research perspective, existing hardware platforms are limited: they are often optimized for concrete, narrow use-cases and, therefore lack the flexibility needed to explore other applications and configurations. We show that a research group can design and build a more general, open, and affordable hardware platform for hybrid systems research. The platform, Enzian, is capable of duplicating the functionality of existing CPU/FPGA systems with comparable performance but in an open, flexible system. It couples a large FPGA with a server-class CPU in an asymmetric cache-coherent NUMA system. Enzian also enables research not possible with existing hybrid platforms, through explicit access to coherence messages, extensive thermal and power instrumentation, and an open, programmable baseboard management processor. Enzian is already being used in multiple projects, is open source (both hardware and software), and available for remote use. We present the design principles of Enzian, the challenges in building it, and evaluate it with a range of existing research use-cases alongside other, more specialized platforms, as well as demonstrating research not possible on existing platforms. @InProceedings{ASPLOS22p434, author = {David Cock and Abishek Ramdas and Daniel Schwyn and Michael Giardino and Adam Turowski and Zhenhao He and Nora Hossle and Dario Korolija and Melissa Licciardello and Kristina Martsenko and Reto Achermann and Gustavo Alonso and Timothy Roscoe}, title = {Enzian: An Open, General, CPU/FPGA Platform for Systems Software Research}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {434--451}, doi = {10.1145/3503222.3507742}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Cohen, Noam	ASPLOS '22: "FlexDriver: A Network Driver ..." FlexDriver: A Network Driver for Your Accelerator Haggai Eran, Maxim Fudim, Gabi Malka, Gal Shalom, Noam Cohen, Amit Hermony, Dotan Levi, Liran Liss, and Mark Silberstein (NVIDIA, Israel; Technion, Israel) We propose a new system design for connecting hardware and FPGA accelerators to the network, allowing the accelerator to directly control commodity Network Interface Cards (NICs) without using the CPU. This enables us to solve the key challenge of leveraging existing NIC hardware offloads such as virtualization, tunneling, and RDMA for accelerator networking. Our approach supports a diverse set of use cases, from direct network access for disaggregated accelerators to inline-acceleration of the network stack, all without the complex networking logic in the accelerator. To demonstrate the feasibility of this approach, we build FlexDriver (FLD), an on-accelerator hardware module that implements a NIC data-plane driver. Our main technical contribution is a mechanism that compresses the NIC control structures by two orders of magnitude, allowing FLD to achieve high networking scalability with low die area cost and no bandwidth interference with the accelerator logic. The prototype for NVIDIA Innova-2 FPGA SmartNICs showcases our design’s utility for three different accelerators: a disaggregated LTE cipher, an IP-defragmentation inline accelerator, and an IoT cryptographic-token authentication offload. These accelerators reach 25 Gbps line rate and leverage the NIC for RDMA processing, VXLAN tunneling, and traffic shaping without CPU involvement. @InProceedings{ASPLOS22p1115, author = {Haggai Eran and Maxim Fudim and Gabi Malka and Gal Shalom and Noam Cohen and Amit Hermony and Dotan Levi and Liran Liss and Mark Silberstein}, title = {FlexDriver: A Network Driver for Your Accelerator}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1115--1129}, doi = {10.1145/3503222.3507776}, year = {2022}, } Publisher's Version
	Crankshaw, Daniel	ASPLOS '22: "SOL: Safe On-Node Learning ..." SOL: Safe On-Node Learning in Cloud Platforms Yawen Wang, Daniel Crankshaw, Neeraja J. Yadwadkar, Daniel Berger, Christos Kozyrakis, and Ricardo Bianchini (Stanford University, USA; Microsoft Research, USA; University of Texas at Austin, USA) Cloud platforms run many software agents on each server node. These agents manage all aspects of node operation, and in some cases frequently collect data and make decisions. Unfortunately, their behavior is typically based on pre-defined static heuristics or offline analysis; they do not leverage on-node machine learning (ML). In this paper, we first characterize the spectrum of node agents in Azure, and identify the classes of agents that are most likely to benefit from on-node ML. We then propose SOL, an extensible framework for designing ML-based agents that are safe and robust to the range of failure conditions that occur in production. SOL provides a simple API to agent developers and manages the scheduling and running of the agent-specific functions they write. We illustrate the use of SOL by implementing three ML-based agents that manage CPU cores, node power, and memory placement. Our experiments show that (1) ML substantially improves our agents, and (2) SOL ensures that agents operate safely under a variety of failure conditions. We conclude that ML-based agents show significant potential and that SOL can help build them. @InProceedings{ASPLOS22p622, author = {Yawen Wang and Daniel Crankshaw and Neeraja J. Yadwadkar and Daniel Berger and Christos Kozyrakis and Ricardo Bianchini}, title = {SOL: Safe On-Node Learning in Cloud Platforms}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {622--634}, doi = {10.1145/3503222.3507704}, year = {2022}, } Publisher's Version
	Cristal, Adrián	ASPLOS '22: "BiSon-e: A Lightweight and ..." BiSon-e: A Lightweight and High-Performance Accelerator for Narrow Integer Linear Algebra Computing on the Edge Enrico Reggiani, Cristóbal Ramírez Lazo, Roger Figueras Bagué, Adrián Cristal, Mauro Olivieri, and Osman Sabri Unsal (Polytechnic University of Catalonia, Spain; Barcelona Supercomputing Center, Spain; Sapienza University of Rome, Italy) Linear algebra computational kernels based on byte and sub-byte integer data formats are at the base of many classes of applications, ranging from Deep Learning to Pattern Matching. Porting the computation of these applications from cloud to edge and mobile devices would enable significant improvements in terms of security, safety, and energy efficiency. However, despite their low memory and energy demands, their intrinsically high computational intensity makes the execution of these workloads challenging on highly resource-constrained devices. In this paper, we present BiSon-e, a novel RISC-V based architecture that accelerates linear algebra kernels based on narrow integer computations on edge processors by performing Single Instruction Multiple Data (SIMD) operations on off-the-shelf scalar Functional Units (FUs). Our novel architecture is built upon the binary segmentation technique, which allows to significantly reduce the memory footprint and the arithmetic intensity of linear algebra kernels requiring narrow data sizes. We integrate BiSon-e into a complete System-on-Chip (SoC) based on RISC-V, synthesized and Place&Routed in 65nm and 22nm technologies, introducing a negligible 0.07% area overhead with respect to the baseline architecture. Our experimental evaluation shows that, when computing the Convolution and Fully-Connected layers of the AlexNet and VGG-16 Convolutional Neural Networks (CNNs) with 8-, 4-, and 2-bit, our solution gains up to 5.6×, 13.9× and 24× in execution time compared to the scalar implementation of a single RISC-V core, and improves the energy efficiency of string matching tasks by 5× when compared to a RISC-V-based Vector Processing Unit (VPU). @InProceedings{ASPLOS22p56, author = {Enrico Reggiani and Cristóbal Ramírez Lazo and Roger Figueras Bagué and Adrián Cristal and Mauro Olivieri and Osman Sabri Unsal}, title = {BiSon-e: A Lightweight and High-Performance Accelerator for Narrow Integer Linear Algebra Computing on the Edge}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {56--69}, doi = {10.1145/3503222.3507746}, year = {2022}, } Publisher's Version
	Cuevas, Michael	ASPLOS '22: "CARAT CAKE: Replacing Paging ..." CARAT CAKE: Replacing Paging via Compiler/Kernel Cooperation Brian Suchy, Souradip Ghosh, Drew Kersnar, Siyuan Chai, Zhen Huang, Aaron Nelson, Michael Cuevas, Alex Bernat, Gaurav Chaudhary, Nikos Hardavellas, Simone Campanoni, and Peter Dinda (Northwestern University, USA) Virtual memory, specifically paging, is undergoing significant innovation due to being challenged by new demands from modern workloads. Recent work has demonstrated an alternative software only design that can result in simplified hardware requirements, even supporting purely physical addressing. While we have made the case for this Compiler- And Runtime-based Address Translation (CARAT) concept, its evaluation was based on a user-level prototype. We now report on incorporating CARAT into a kernel, forming Compiler- And Runtime-based Address Translation for CollAborative Kernel Environments (CARAT CAKE). In our implementation, a Linux-compatible x64 process abstraction can be based either on CARAT CAKE, or on a sophisticated paging implementation. Implementing CARAT CAKE involves kernel changes and compiler optimizations/transformations that must work on all code in the system, including kernel code. We evaluate CARAT CAKE in comparison with paging and find that CARAT CAKE is able to achieve the functionality of paging (protection, mapping, and movement properties) with minimal overhead. In turn, CARAT CAKE allows significant new benefits for systems including energy savings, larger L1 caches, and arbitrary granularity memory management. @InProceedings{ASPLOS22p98, author = {Brian Suchy and Souradip Ghosh and Drew Kersnar and Siyuan Chai and Zhen Huang and Aaron Nelson and Michael Cuevas and Alex Bernat and Gaurav Chaudhary and Nikos Hardavellas and Simone Campanoni and Peter Dinda}, title = {CARAT CAKE: Replacing Paging via Compiler/Kernel Cooperation}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {98--114}, doi = {10.1145/3503222.3507771}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available
	Cui, Heming	ASPLOS '22: "NASPipe: High Performance ..." NASPipe: High Performance and Reproducible Pipeline Parallel Supernet Training via Causal Synchronous Parallelism Shixiong Zhao, Fanxin Li, Xusheng Chen, Tianxiang Shen, Li Chen, Sen Wang, Nicholas Zhang, Cheng Li, and Heming Cui (University of Hong Kong, China; Huawei Technologies, China; University of Science and Technology of China, China) Supernet training, a prevalent and important paradigm in Neural Architecture Search, embeds the whole DNN architecture search space into one monolithic supernet, iteratively activates a subset of the supernet (i.e., a subnet) for fitting each batch of data, and searches a high-quality subnet which meets specific requirements. Although training subnets in parallel on multiple GPUs is desirable for acceleration, there inherently exists a race hazard that concurrent subnets may access the same DNN layers. Existing systems support neither efficiently parallelizing subnets’ training executions, nor resolving the race hazard deterministically, leading to unreproducible training procedures and potentiallly non-trivial accuracy loss. We present NASPipe, the first high-performance and reproducible distributed supernet training system via causal synchronous parallel (CSP) pipeline scheduling abstraction: NASPipe partitions a supernet across GPUs and concurrently executes multiple generated sub-tasks (subnets) in a pipelined manner; meanwhile, it oversees the correlations between the subnets and deterministically resolves any causal dependency caused by subnets’ layer sharing. To obtain high performance, NASPipe’s CSP scheduler exploits the fact that the larger a supernet spans, the fewer dependencies manifest between chronologically close subnets; therefore, it aggressively schedules the subnets with larger chronological orders into execution, only if they are not causally dependent on unfinished precedent subnets. Moreover, to relieve the excessive GPU memory burden for holding the whole supernet’s parameters, NASPipe uses a context switch technique that stashes the whole supernet in CPU memory, precisely predicts the subnets’ schedule, and pre-fetches/evicts a subnet before/after its execution. The evaluation shows that NASPipe is the only system that retains supernet training reproducibility, while achieving a comparable and even higher performance (up to 7.8X) compared to three recent pipeline training systems (e.g., GPipe). @InProceedings{ASPLOS22p374, author = {Shixiong Zhao and Fanxin Li and Xusheng Chen and Tianxiang Shen and Li Chen and Sen Wang and Nicholas Zhang and Cheng Li and Heming Cui}, title = {NASPipe: High Performance and Reproducible Pipeline Parallel Supernet Training via Causal Synchronous Parallelism}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {374--387}, doi = {10.1145/3503222.3507735}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Dadu, Vidushi	ASPLOS '22: "TaskStream: Accelerating Task-Parallel ..." TaskStream: Accelerating Task-Parallel Workloads by Recovering Program Structure Vidushi Dadu and Tony Nowatzki (University of California at Los Angeles, USA) Reconfigurable accelerators, like CGRAs and dataflow architectures, have come to prominence for addressing data-processing problems. However, they are largely limited to workloads with regular parallelism, precluding their applicability to prevalent task-parallel workloads. Reconfigurable architectures and task parallelism seem to be at odds, as the former requires repetitive and simple program structure, and the latter breaks program structure to create small, individually scheduled program units. Our insight is that if tasks and their potential for communication structure are first-class primitives in the hardware, it is possible to recover program structure with extremely low overhead. We propose a task execution model for accelerators called TaskStream, which annotates task dependences with information sufficient to recover inter-task structure. TaskStream enables work-aware load balancing, recovery of pipelined inter-task dependences, and recovery of inter-task read sharing through multicasting. We apply TaskStream to a reconfigurable dataflow architecture, creating a seamless hierarchical dataflow model for task-parallel workloads. We compare our accelerator, Delta, with an equivalent static-parallel design. Overall, we find that our execution model can improve performance by 2.2× with only 3.6% area overhead, while alleviating the programming burden of managing task distribution. @InProceedings{ASPLOS22p1, author = {Vidushi Dadu and Tony Nowatzki}, title = {TaskStream: Accelerating Task-Parallel Workloads by Recovering Program Structure}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1--13}, doi = {10.1145/3503222.3507706}, year = {2022}, } Publisher's Version
	Dai, Guohao	ASPLOS '22: "A One-for-All and O(V ..." A One-for-All and O(V log(V ))-Cost Solution for Parallel Merge Style Operations on Sorted Key-Value Arrays Bangyan Wang, Lei Deng, Fei Sun, Guohao Dai, Liu Liu, Yu Wang, and Yuan Xie (University of California at Santa Barbara, USA; Tsinghua University, China; Alibaba DAMO Academy, China) The processing of sorted key-value arrays using a “merge style operation (MSO)” is a very basic and important problem in domains like scientific computing, deep learning, database, graph analysis, sorting, set-operation etc. MSOs dominate the execution time in some important applications like SpGEMM and graph mining. For example, sparse vector addition as an MSO takes up to 98% execution time in SpGEMM in our experiment. For this reason, accelerating MSOs on CPU, GPU, and accelerators using parallel execution has been extensively studied but the solutions in prior work have three major limitations. (1) They treat different MSOs as isolated problems using incompatible methods and an unified solution is still lacking. (2) They do not have the flexibility to support variable key/value sizes and value calculations in the runtime given a fixed hardware design. (3) They require a quadratic hardware cost (O(V2)) for given parallelism V in most cases. To address above three limitations, we make the following efforts. (1) We present a one-for-all solution to support all interested MSOs based on a unified abstraction model “restricted zip machine (RZM)”. (2) We propose a set of composable and parallel primitives for RZM to provide the flexibility to support variable key/value sizes and value calculations. (3) We provide the hardware design to implement the proposed primitives using only O(Vlog(V)) resource. With the above techniques, a flexible and efficient solution for MSOs has been built. Our design can be used either as a drop-in replacement of the merge unit in prior accelerators to reduce the cost from O(V2) to O(Vlog(V)), or as an extension to the SIMD ISA of CPU and GPU. In our evaluation on CPU, when V=16 (512-bit SIMD, 32-bit element), we achieve significant speedup on a range of representative kernels including set operations (8.4×), database joins (7.3×), sparse vector/matrix/tensor addition/multiplication on real/complex numbers (6.5×), merge sort (8.0× over scalar, 3.4× over the state-of-the-art SIMD), and SpGEMM (4.4× over the best one in the baseline collection). @InProceedings{ASPLOS22p669, author = {Bangyan Wang and Lei Deng and Fei Sun and Guohao Dai and Liu Liu and Yu Wang and Yuan Xie}, title = {A One-for-All and <i>O</i>(<i>V</i> log(<i>V</i> ))-Cost Solution for Parallel Merge Style Operations on Sorted Key-Value Arrays}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {669--682}, doi = {10.1145/3503222.3507728}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Dang, Zheng	ASPLOS '22: "NVAlloc: Rethinking Heap Metadata ..." NVAlloc: Rethinking Heap Metadata Management in Persistent Memory Allocators Zheng Dang, Shuibing He, Peiyi Hong, Zhenxin Li, Xuechen Zhang, Xian-He Sun, and Gang Chen (Zhejiang University, China; Washington State University, USA; Illinois Institute of Technology, USA) Persistent memory allocation is a fundamental building block for developing high-performance and in-memory applications. Existing persistent memory allocators suffer from suboptimal heap organizations that introduce repeated cache line flushes and small random accesses in persistent memory. Worse, many allocators use static slab segregation resulting in a dramatic increase in memory consumption when allocation request size is changed. In this paper, we design a novel allocator, named NVAlloc, to solve the above issues simultaneously. First, NVAlloc eliminates cache line reflushes by mapping contiguous data blocks in slabs to interleaved metadata entries stored in different cache lines. Second, it writes small metadata units to a persistent bookkeeping log in a sequential pattern to remove random heap metadata accesses in persistent memory. Third, instead of using static slab segregation, it supports slab morphing, which allows slabs to be transformed between size classes to significantly improve slab usage. NVAlloc is complementary to the existing consistency models. Results on 6 benchmarks demonstrate that NVAlloc improves the performance of state-of-the-art persistent memory allocators by up to 6.4x and 57x for small and large allocations, respectively. Using NVAlloc reduces memory usage by up to 57.8%. Besides, we integrate NVAlloc in a persistent FPTree. Compared to the state-of-the-art allocators, NVAlloc improves the performance of this application by up to 3.1x. @InProceedings{ASPLOS22p115, author = {Zheng Dang and Shuibing He and Peiyi Hong and Zhenxin Li and Xuechen Zhang and Xian-He Sun and Gang Chen}, title = {NVAlloc: Rethinking Heap Metadata Management in Persistent Memory Allocators}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {115--127}, doi = {10.1145/3503222.3507743}, year = {2022}, } Publisher's Version
	Das, Poulami	ASPLOS '22: "HAMMER: Boosting Fidelity ..." HAMMER: Boosting Fidelity of Noisy Quantum Circuits by Exploiting Hamming Behavior of Erroneous Outcomes Swamit Tannu, Poulami Das, Ramin Ayanzadeh, and Moinuddin Qureshi (University of Wisconsin-Madison, USA; Georgia Institute of Technology, USA) Quantum computers with hundreds of qubits will be available soon. Unfortunately, high device error-rates pose a significant challenge in using these near-term quantum systems to power real-world applications. Executing a program on existing quantum systems generates both correct and incorrect outcomes, but often, the output distribution is too noisy to distinguish between them. In this paper, we show that erroneous outcomes are not arbitrary but exhibit a well-defined structure when represented in the Hamming space. Our experiments on IBM and Google quantum computers show that the most frequent erroneous outcomes are more likely to be close in the Hamming space to the correct outcome. We exploit this behavior to improve the ability to infer the correct outcome. We propose Hamming Reconstruction (HAMMER), a post-processing technique that leverages the observation of Hamming behavior to reconstruct the noisy output distribution, such that the resulting distribution has higher fidelity. We evaluate HAMMER using experimental data from Google and IBM quantum computers with more than 500 unique quantum circuits and obtain an average improvement of 1.37x in the quality of solution. On Google’s publicly available QAOA datasets, we show that HAMMER sharpens the gradients on the cost function landscape. @InProceedings{ASPLOS22p529, author = {Swamit Tannu and Poulami Das and Ramin Ayanzadeh and Moinuddin Qureshi}, title = {HAMMER: Boosting Fidelity of Noisy Quantum Circuits by Exploiting Hamming Behavior of Erroneous Outcomes}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {529--540}, doi = {10.1145/3503222.3507703}, year = {2022}, } Publisher's Version ASPLOS '22: "LILLIPUT: A Lightweight Low-Latency ..." LILLIPUT: A Lightweight Low-Latency Lookup-Table Decoder for Near-Term Quantum Error Correction Poulami Das, Aditya Locharla, and Cody Jones (Georgia Institute of Technology, USA; Google, USA) The error rates of quantum devices are orders of magnitude higher than what is needed to run most quantum applications. To close this gap, Quantum Error Correction (QEC) encodes logical qubits and distributes information using several physical qubits. By periodically executing a syndrome extraction circuit on the logical qubits, information about errors (called syndrome) is extracted while running programs. A decoder uses these syndromes to identify and correct errors in real time, which is necessary to prevent accumulation of errors. Unfortunately, software decoders are slow and hardware decoders are fast but less accurate. Thus, almost all QEC studies so far have relied on offline decoding. To enable real-time decoding in near-term QEC, we propose LILLIPUT– a Lightweight Low Latency Look-Up Table decoder. LILLIPUT consists of two parts– First, it translates syndromes into error detection events that index into a Look-Up Table (LUT) whose entry provides the error information in real-time. Second, it programs the LUTs with error assignments for all possible error events by running a software decoder offline. LILLIPUT tolerates an error on any operation in the quantum hardware, including gates and measurements, and the number of tolerated errors grows with the size of the code. LILLIPUT utilizes less than 7% logic on off-the-shelf FPGAs enabling practical adoption, as FPGAs are already used to design the control and readout circuits in existing systems. LILLIPUT incurs a latency of a few nanoseconds and enables real-time decoding. We also propose Compressed LUTs (CLUTs) to reduce the memory required by LILLIPUT. By exploiting the fact that not all error events are equally likely and only storing data for the most probable error events, CLUTs reduce the memory needed by up-to 107x (from 148 MB to 1.38 MB) without degrading the accuracy. @InProceedings{ASPLOS22p541, author = {Poulami Das and Aditya Locharla and Cody Jones}, title = {LILLIPUT: A Lightweight Low-Latency Lookup-Table Decoder for Near-Term Quantum Error Correction}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {541--553}, doi = {10.1145/3503222.3507707}, year = {2022}, } Publisher's Version
	DeHon, André	ASPLOS '22: "PLD: Fast FPGA Compilation ..." PLD: Fast FPGA Compilation to Make Reconfigurable Acceleration Compatible with Modern Incremental Refinement Software Development Yuanlong Xiao, Eric Micallef, Andrew Butt, Matthew Hofmann, Marc Alston, Matthew Goldsmith, Andrew Merczynski-Hait, and André DeHon (University of Pennsylvania, USA) FPGA-based accelerators are demonstrating significant absolute performance and energy efficiency compared with general-purpose CPUs. While FPGA computations can now be described in standard, programming languages, like C, development for FPGAs accelerators remains tedious and inaccessible to modern software engineers. Slow compiles (potentially taking tens of hours) inhibit the rapid, incremental refinement of designs that is the hallmark of modern software engineering. To address this issue, we introduce separate compilation and linkage into the FPGA design flow, providing faster design turns more familiar to software development. To realize this flow, we provide abstractions, compiler options, and compiler flow that allow the same C source code to be compiled to processor cores in seconds and to FPGA regions in minutes, providing the missing -O0 and -O1 options familiar in software development. This raises the FPGA programming level and standardizes the programming experience, bringing FPGA-based accelerators into a more familiar software platform ecosystem for software engineers. @InProceedings{ASPLOS22p933, author = {Yuanlong Xiao and Eric Micallef and Andrew Butt and Matthew Hofmann and Marc Alston and Matthew Goldsmith and Andrew Merczynski-Hait and André DeHon}, title = {PLD: Fast FPGA Compilation to Make Reconfigurable Acceleration Compatible with Modern Incremental Refinement Software Development}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {933--945}, doi = {10.1145/3503222.3507740}, year = {2022}, } Publisher's Version
	De Jong, Wibe	ASPLOS '22: "QUEST: Systematically Approximating ..." QUEST: Systematically Approximating Quantum Circuits for Higher Output Fidelity Tirthak Patel, Ed Younis, Costin Iancu, Wibe de Jong, and Devesh Tiwari (Northeastern University, USA; Lawrence Berkeley National Laboratory, USA) We present QUEST, a procedure to systematically generate approximations for quantum circuits to reduce their CNOT gate count. Our approach employs circuit partitioning for scalability with procedures to 1) reduce circuit length using approximate synthesis, 2) improve fidelity by running circuits that represent key samples in the approximation space, and 3) reason about approximation upper bound. Our evaluation results indicate that our approach of "dissimilar" approximations provides close fidelity to the original circuit. Overall, the results indicate that QUEST can reduce CNOT gate count by 30-80% on ideal systems and decrease the impact of noise on existing and near-future quantum systems. @InProceedings{ASPLOS22p514, author = {Tirthak Patel and Ed Younis and Costin Iancu and Wibe de Jong and Devesh Tiwari}, title = {QUEST: Systematically Approximating Quantum Circuits for Higher Output Fidelity}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {514--528}, doi = {10.1145/3503222.3507739}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional
	Demsky, Brian	ASPLOS '22: "Yashme: Detecting Persistency ..." Yashme: Detecting Persistency Races Hamed Gorjiara, Guoqing Harry Xu, and Brian Demsky (University of California at Irvine, USA; University of California at Los Angeles, USA) Persistent memory (PM) or Non-Volatile Random-Access Memory (NVRAM) hardware such as Intel’s Optane memory product promises to transform how programs store and manipulate information. Ensuring that persistent memory programs are crash consistent is a major challenge. We present a novel class of crash consistency bugs for persistent memory programs, which we call persistency races. Persistency races can cause non-atomic stores to be made partially persistent. Persistency races arise due to the interaction of standard compiler optimizations with persistent memory semantics. We present Yashme, the first detector for persistency races. A major challenge is that in order to detect persistency races, the execution must crash in a very narrow window between a store with a persistency race and its corresponding cache flush operation, making it challenging for naïve techniques to be effective. Yashme overcomes this challenge with a novel technique for detecting races in executions that are prefixes of the pre-crash execution. This technique enables Yashme to effectively find persistency races even if the injected crashes do not fall into that window. We have evaluated Yashme on a range of persistent memory benchmarks and have found 26 real persistency races that have never been reported before. @InProceedings{ASPLOS22p830, author = {Hamed Gorjiara and Guoqing Harry Xu and Brian Demsky}, title = {Yashme: Detecting Persistency Races}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {830--845}, doi = {10.1145/3503222.3507766}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Deng, Lei	ASPLOS '22: "A One-for-All and O(V ..." A One-for-All and O(V log(V ))-Cost Solution for Parallel Merge Style Operations on Sorted Key-Value Arrays Bangyan Wang, Lei Deng, Fei Sun, Guohao Dai, Liu Liu, Yu Wang, and Yuan Xie (University of California at Santa Barbara, USA; Tsinghua University, China; Alibaba DAMO Academy, China) The processing of sorted key-value arrays using a “merge style operation (MSO)” is a very basic and important problem in domains like scientific computing, deep learning, database, graph analysis, sorting, set-operation etc. MSOs dominate the execution time in some important applications like SpGEMM and graph mining. For example, sparse vector addition as an MSO takes up to 98% execution time in SpGEMM in our experiment. For this reason, accelerating MSOs on CPU, GPU, and accelerators using parallel execution has been extensively studied but the solutions in prior work have three major limitations. (1) They treat different MSOs as isolated problems using incompatible methods and an unified solution is still lacking. (2) They do not have the flexibility to support variable key/value sizes and value calculations in the runtime given a fixed hardware design. (3) They require a quadratic hardware cost (O(V2)) for given parallelism V in most cases. To address above three limitations, we make the following efforts. (1) We present a one-for-all solution to support all interested MSOs based on a unified abstraction model “restricted zip machine (RZM)”. (2) We propose a set of composable and parallel primitives for RZM to provide the flexibility to support variable key/value sizes and value calculations. (3) We provide the hardware design to implement the proposed primitives using only O(Vlog(V)) resource. With the above techniques, a flexible and efficient solution for MSOs has been built. Our design can be used either as a drop-in replacement of the merge unit in prior accelerators to reduce the cost from O(V2) to O(Vlog(V)), or as an extension to the SIMD ISA of CPU and GPU. In our evaluation on CPU, when V=16 (512-bit SIMD, 32-bit element), we achieve significant speedup on a range of representative kernels including set operations (8.4×), database joins (7.3×), sparse vector/matrix/tensor addition/multiplication on real/complex numbers (6.5×), merge sort (8.0× over scalar, 3.4× over the state-of-the-art SIMD), and SpGEMM (4.4× over the best one in the baseline collection). @InProceedings{ASPLOS22p669, author = {Bangyan Wang and Lei Deng and Fei Sun and Guohao Dai and Liu Liu and Yu Wang and Yuan Xie}, title = {A One-for-All and <i>O</i>(<i>V</i> log(<i>V</i> ))-Cost Solution for Parallel Merge Style Operations on Sorted Key-Value Arrays}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {669--682}, doi = {10.1145/3503222.3507728}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Deutsch, Peter W.	ASPLOS '22: "DAGguise: Mitigating Memory ..." DAGguise: Mitigating Memory Timing Side Channels Peter W. Deutsch, Yuheng Yang, Thomas Bourgeat, Jules Drean, Joel S. Emer, and Mengjia Yan (Massachusetts Institute of Technology, USA; NVIDIA, USA) This paper studies the mitigation of memory timing side channels, where attackers utilize contention within DRAM controllers to infer a victim’s secrets. Already practical, this class of channels poses an important challenge to secure computing in shared memory environments. Existing state-of-the-art memory timing side channel mitigations have several key performance and security limitations. Prior schemes require onerous static bandwidth partitioning, extensive profiling phases, or simply fail to protect against attacks which exploit fine-grained timing and bank information. We present DAGguise, a defense mechanism which fully protects against memory timing side channels while allowing for dynamic traffic contention in order to achieve good performance. DAGguise utilizes a novel abstract memory access representation, the Directed Acyclic Request Graph (rDAG for short), to model memory access patterns which experience contention. DAGguise shapes a victim’s memory access patterns according to a publicly known rDAG obtained through a lightweight profiling stage, completely eliminating information leakage. We formally verify the security of DAGguise, proving that it maintains strong security guarantees. Moreover, by allowing dynamic traffic contention, DAGguise achieves a 12% overall system speedup relative to Fixed Service, which is the state-of-the-art mitigation mechanism, with up to a 20% relative speedup for co-located applications which do not require protection. We further claim that the principles of DAGguise can be generalized to protect against other types of scheduler-based timing side channels, such as those targeting on-chip networks, or functional units in SMT cores. @InProceedings{ASPLOS22p329, author = {Peter W. Deutsch and Yuheng Yang and Thomas Bourgeat and Jules Drean and Joel S. Emer and Mengjia Yan}, title = {DAGguise: Mitigating Memory Timing Side Channels}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {329--343}, doi = {10.1145/3503222.3507747}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Devecsery, David	ASPLOS '22: "Creating Concise and Efficient ..." Creating Concise and Efficient Dynamic Analyses with ALDA Xiang Cheng and David Devecsery (Georgia Institute of Technology, USA) Dynamic program analyses are essential to creating safe, reliable, and productive computing environments. However, these analyses are challenging and time-consuming to construct due to the low-level optimization required to achieve acceptable performance. Consequently, many analyses are often never realized, or have inefficient implementations. In this work we argue that many analyses can and should be constructed with a high-level description language, leaving the burden of low-level optimizations to the analysis instrumentation system itself. We propose a novel language for dynamic analysis called ALDA. ALDA leverages common structuring of dynamic analyses to provide a simple and high-level description for dynamic analyses. Through restricting the supported behaviors to only essential operations in dynamic analyses, an optimizing compiler for ALDA can create analysis implementations with performance on-par to hand-tuned analyses. To demonstrate ALDA’s universality and efficiency, we create an optimizing compiler for ALDA targeting the LLVM instrumentation framework named ALDAcc. We use ALDAcc to construct 8 different dynamic analysis algorithms, including the popular MemorySanitizer analysis, and show their construction is succinct and simple. By comparing two of them (Eraser and MemorySanitizer) with their hand-tuned implementations, we show that ALDAcc’s optimized analyses are comparable to hand-tuned implementations. @InProceedings{ASPLOS22p740, author = {Xiang Cheng and David Devecsery}, title = {Creating Concise and Efficient Dynamic Analyses with ALDA}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {740--752}, doi = {10.1145/3503222.3507760}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Dhakshinamurthy, Saravanan	ASPLOS '22: "IOCost: Block IO Control for ..." IOCost: Block IO Control for Containers in Datacenters Tejun Heo, Dan Schatzberg, Andrew Newell, Song Liu, Saravanan Dhakshinamurthy, Iyswarya Narayanan, Josef Bacik, Chris Mason, Chunqiang Tang, and Dimitrios Skarlatos (Meta, USA; Carnegie Mellon University, USA) Resource isolation is a fundamental requirement in datacenter environments. However, our production experience in Meta’s large-scale datacenters shows that existing IO control mechanisms for block storage are inadequate in containerized environments. IO control needs to provide proportional resources to containers while taking into account the hardware heterogeneity of storage devices and the idiosyncrasies of the workloads deployed in datacenters. The speed of modern SSDs requires IO control to execute with low-overheads. Furthermore, IO control should strive for work conservation, take into account the interactions with the memory management subsystem, and avoid priority inversions that lead to isolation failures. To address these challenges, this paper presents IOCost, an IO control solution that is designed for containerized environments and provides scalable, work-conserving, and low-overhead IO control for heterogeneous storage devices and diverse workloads in datacenters. IOCost performs offline profiling to build a device model and uses it to estimate device occupancy of each IO request. To minimize runtime overhead, it separates IO control into a fast per-IO issue path and a slower periodic planning path. A novel work-conserving budget donation algorithm enables containers to dynamically share unused budget. We have deployed IOCost across the entirety of Meta’s datacenters comprised of millions of ma- chines, upstreamed IOCost to the Linux kernel, and open-sourced our device-profiling tools. IOCost has been running in production for two years, providing IO control for Meta’s fleet. We describe the design of IOCost and share our experience deploying it at scale. @InProceedings{ASPLOS22p595, author = {Tejun Heo and Dan Schatzberg and Andrew Newell and Song Liu and Saravanan Dhakshinamurthy and Iyswarya Narayanan and Josef Bacik and Chris Mason and Chunqiang Tang and Dimitrios Skarlatos}, title = {IOCost: Block IO Control for Containers in Datacenters}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {595--608}, doi = {10.1145/3503222.3507727}, year = {2022}, } Publisher's Version
	Dinda, Peter	ASPLOS '22: "CARAT CAKE: Replacing Paging ..." CARAT CAKE: Replacing Paging via Compiler/Kernel Cooperation Brian Suchy, Souradip Ghosh, Drew Kersnar, Siyuan Chai, Zhen Huang, Aaron Nelson, Michael Cuevas, Alex Bernat, Gaurav Chaudhary, Nikos Hardavellas, Simone Campanoni, and Peter Dinda (Northwestern University, USA) Virtual memory, specifically paging, is undergoing significant innovation due to being challenged by new demands from modern workloads. Recent work has demonstrated an alternative software only design that can result in simplified hardware requirements, even supporting purely physical addressing. While we have made the case for this Compiler- And Runtime-based Address Translation (CARAT) concept, its evaluation was based on a user-level prototype. We now report on incorporating CARAT into a kernel, forming Compiler- And Runtime-based Address Translation for CollAborative Kernel Environments (CARAT CAKE). In our implementation, a Linux-compatible x64 process abstraction can be based either on CARAT CAKE, or on a sophisticated paging implementation. Implementing CARAT CAKE involves kernel changes and compiler optimizations/transformations that must work on all code in the system, including kernel code. We evaluate CARAT CAKE in comparison with paging and find that CARAT CAKE is able to achieve the functionality of paging (protection, mapping, and movement properties) with minimal overhead. In turn, CARAT CAKE allows significant new benefits for systems including energy savings, larger L1 caches, and arbitrary granularity memory management. @InProceedings{ASPLOS22p98, author = {Brian Suchy and Souradip Ghosh and Drew Kersnar and Siyuan Chai and Zhen Huang and Aaron Nelson and Michael Cuevas and Alex Bernat and Gaurav Chaudhary and Nikos Hardavellas and Simone Campanoni and Peter Dinda}, title = {CARAT CAKE: Replacing Paging via Compiler/Kernel Cooperation}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {98--114}, doi = {10.1145/3503222.3507771}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available
	Ding, Luochangqi	ASPLOS '22: "Efficiently Detecting Concurrency ..." Efficiently Detecting Concurrency Bugs in Persistent Memory Programs Zhangyu Chen, Yu Hua, Yongle Zhang, and Luochangqi Ding (Huazhong University of Science and Technology, China; Purdue University, USA) Due to the salient DRAM-comparable performance, TB-scale capacity, and non-volatility, persistent memory (PM) provides new opportunities for large-scale in-memory computing with instant crash recovery. However, programming PM systems is error-prone due to the existence of crash-consistency bugs, which are challenging to diagnose especially with concurrent programming widely adopted in PM applications to exploit hardware parallelism. Existing bug detection tools for DRAM-based concurrency issues cannot detect PM crash-consistency bugs because they are oblivious to PM operations and PM consistency. On the other hand, existing PM-specific debugging tools only focus on sequential PM programs and cannot effectively detect crash-consistency issues hidden in concurrent executions. In order to effectively detect crash-consistency bugs that only manifest in concurrent executions, we propose PMRace, the first PM-specific concurrency bug detection tool. We identify and define two new types of concurrent crash-consistency bugs: PM Inter-thread Inconsistency and PM Synchronization Inconsistency. In particular, PMRace adopts PM-aware and coverage-guided fuzz testing to explore PM program executions. For PM Inter-thread Inconsistency, which denotes the data inconsistency hidden in thread interleavings, PMRace performs PM-aware interleaving exploration and thread scheduling to drive the execution towards executions that reveal such inconsistencies. For PM Synchronization Inconsistency between persisted synchronization variables and program data, PMRace identifies the inconsistency during interleaving exploration. The post-failure validation reduces the false positives that come from custom crash recovery mechanisms. PMRace has found 14 bugs (10 new bugs) in real-world concurrent PM systems including PM-version memcached. @InProceedings{ASPLOS22p873, author = {Zhangyu Chen and Yu Hua and Yongle Zhang and Luochangqi Ding}, title = {Efficiently Detecting Concurrency Bugs in Persistent Memory Programs}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {873--887}, doi = {10.1145/3503222.3507755}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Ding, Yufei	ASPLOS '22: "DOTA: Detect and Omit Weak ..." DOTA: Detect and Omit Weak Attentions for Scalable Transformer Acceleration Zheng Qu, Liu Liu, Fengbin Tu, Zhaodong Chen, Yufei Ding, and Yuan Xie (University of California at Santa Barbara, USA) Transformer Neural Networks have demonstrated leading performance in many applications spanning over language understanding, image processing, and generative modeling. Despite the impressive performance, long-sequence Transformer processing is expensive due to quadratic computation complexity and memory consumption of self-attention. In this paper, we present DOTA, an algorithm-architecture co-design that effectively addresses the challenges of scalable Transformer inference. Based on the insight that not all connections in an attention graph are equally important, we propose to jointly optimize a lightweight Detector with the Transformer model to accurately detect and omit weak connections during runtime. Furthermore, we design a specialized system architecture for end-to-end Transformer acceleration using the proposed attention detection mechanism. Experiments on a wide range of benchmarks demonstrate the superior performance of DOTA over other solutions. In summary, DOTA achieves 152.6x and 4.5x performance speedup and orders of magnitude energy-efficiency improvements over GPU and customized hardware, respectively. @InProceedings{ASPLOS22p14, author = {Zheng Qu and Liu Liu and Fengbin Tu and Zhaodong Chen and Yufei Ding and Yuan Xie}, title = {DOTA: Detect and Omit Weak Attentions for Scalable Transformer Acceleration}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {14--26}, doi = {10.1145/3503222.3507738}, year = {2022}, } Publisher's Version ASPLOS '22: "Paulihedral: A Generalized ..." Paulihedral: A Generalized Block-Wise Compiler Optimization Framework for Quantum Simulation Kernels Gushu Li, Anbang Wu, Yunong Shi, Ali Javadi-Abhari, Yufei Ding, and Yuan Xie (University of California at Santa Barbara, USA; Amazon, USA; IBM, USA) The quantum simulation kernel is an important subroutine appearing as a very long gate sequence in many quantum programs. In this paper, we propose Paulihedral, a block-wise compiler framework that can deeply optimize this subroutine by exploiting high-level program structure and optimization opportunities. Paulihedral first employs a new Pauli intermediate representation that can maintain the high-level semantics and constraints in quantum simulation kernels. This naturally enables new large-scale optimizations that are hard to implement at the low gate-level. In particular, we propose two technology-independent instruction scheduling passes, and two technology-dependent code optimization passes which reconcile the circuit synthesis, gate cancellation, and qubit mapping stages of the compiler. Experimental results show that Paulihedral can outperform state-of-the-art compiler infrastructures in a wide-range of applications on both near-term superconducting quantum processors and future fault-tolerant quantum computers. @InProceedings{ASPLOS22p554, author = {Gushu Li and Anbang Wu and Yunong Shi and Ali Javadi-Abhari and Yufei Ding and Yuan Xie}, title = {Paulihedral: A Generalized Block-Wise Compiler Optimization Framework for Quantum Simulation Kernels}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {554--569}, doi = {10.1145/3503222.3507715}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Doupé, Adam	ASPLOS '22: "ViK: Practical Mitigation ..." ViK: Practical Mitigation of Temporal Memory Safety Violations through Object ID Inspection Haehyun Cho, Jinbum Park, Adam Oest, Tiffany Bao, Ruoyu Wang, Yan Shoshitaishvili, Adam Doupé, and Gail-Joon Ahn (Soongsil University, South Korea; Samsung Research, South Korea; PayPal, USA; Arizona State University, USA; Samsung Research, USA) Temporal memory safety violations, such as use-after-free (UAF) vulnerabilities, are a critical security issue for software written in memory-unsafe languages such as C and C++. In this paper, we introduce ViK, a novel, lightweight, and widely applicable runtime defense that can protect both operating system (OS) kernels and user-space applications against temporal memory safety violations. ViK performs object ID inspection, where it assigns a random identifier to every allocated object and stores the identifier in the unused bits of the corresponding pointer. When the pointer is used, ViK inspects the value of a pointer before dereferencing, ensuring that the pointer still references the original object. To the best of our knowledge, this is the first mitigation against temporal memory safety violations that scales to OS kernels. We evaluated the software prototype of ViK on Android and Linux kernels and observed runtime overhead of around 20%. Also, we evaluated a hardware-assisted prototype of ViK on Android kernel, where the runtime overhead was as low as 2%. @InProceedings{ASPLOS22p271, author = {Haehyun Cho and Jinbum Park and Adam Oest and Tiffany Bao and Ruoyu Wang and Yan Shoshitaishvili and Adam Doupé and Gail-Joon Ahn}, title = {ViK: Practical Mitigation of Temporal Memory Safety Violations through Object ID Inspection}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {271--284}, doi = {10.1145/3503222.3507780}, year = {2022}, } Publisher's Version Artifacts Functional
	Drean, Jules	ASPLOS '22: "DAGguise: Mitigating Memory ..." DAGguise: Mitigating Memory Timing Side Channels Peter W. Deutsch, Yuheng Yang, Thomas Bourgeat, Jules Drean, Joel S. Emer, and Mengjia Yan (Massachusetts Institute of Technology, USA; NVIDIA, USA) This paper studies the mitigation of memory timing side channels, where attackers utilize contention within DRAM controllers to infer a victim’s secrets. Already practical, this class of channels poses an important challenge to secure computing in shared memory environments. Existing state-of-the-art memory timing side channel mitigations have several key performance and security limitations. Prior schemes require onerous static bandwidth partitioning, extensive profiling phases, or simply fail to protect against attacks which exploit fine-grained timing and bank information. We present DAGguise, a defense mechanism which fully protects against memory timing side channels while allowing for dynamic traffic contention in order to achieve good performance. DAGguise utilizes a novel abstract memory access representation, the Directed Acyclic Request Graph (rDAG for short), to model memory access patterns which experience contention. DAGguise shapes a victim’s memory access patterns according to a publicly known rDAG obtained through a lightweight profiling stage, completely eliminating information leakage. We formally verify the security of DAGguise, proving that it maintains strong security guarantees. Moreover, by allowing dynamic traffic contention, DAGguise achieves a 12% overall system speedup relative to Fixed Service, which is the state-of-the-art mitigation mechanism, with up to a 20% relative speedup for co-located applications which do not require protection. We further claim that the principles of DAGguise can be generalized to protect against other types of scheduler-based timing side channels, such as those targeting on-chip networks, or functional units in SMT cores. @InProceedings{ASPLOS22p329, author = {Peter W. Deutsch and Yuheng Yang and Thomas Bourgeat and Jules Drean and Joel S. Emer and Mengjia Yan}, title = {DAGguise: Mitigating Memory Timing Side Channels}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {329--343}, doi = {10.1145/3503222.3507747}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Du, Dong	ASPLOS '22: "Serverless Computing on Heterogeneous ..." Serverless Computing on Heterogeneous Computers Dong Du, Qingyuan Liu, Xueqiang Jiang, Yubin Xia, Binyu Zang, and Haibo Chen (Shanghai Jiao Tong University, China; Shanghai Artificial Intelligence Laboratory, China) Existing serverless computing platforms are built upon homogeneous computers, limiting the function density and restricting serverless computing to limited scenarios. We introduce Molecule, the first serverless computing system utilizing heterogeneous computers. Molecule enables both general-purpose devices (e.g., Nvidia DPU) and domain-specific accelerators (e.g., FPGA and GPU) for serverless applications that significantly improve function density (50% higher) and application performance (up to 34.6x). To achieve these results, we first propose XPU-Shim, a distributed shim to bridge the gap between underlying multi-OS systems (when using general-purpose devices) and our serverless runtime (i.e., Molecule). We further introduce vectorized sandbox, a sandbox abstraction to abstract hardware heterogeneity (when using domain-specific accelerators). Moreover, we also review state-of-the-art serverless optimizations on startup and communication latency and overcome the challenges to implement them on heterogeneous computers. We have implemented Molecule on real platforms with Nvidia DPUs and Xilinx FPGAs and evaluate it using benchmarks and real-world applications. @InProceedings{ASPLOS22p797, author = {Dong Du and Qingyuan Liu and Xueqiang Jiang and Yubin Xia and Binyu Zang and Haibo Chen}, title = {Serverless Computing on Heterogeneous Computers}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {797--813}, doi = {10.1145/3503222.3507732}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Emer, Joel S.	ASPLOS '22: "DAGguise: Mitigating Memory ..." DAGguise: Mitigating Memory Timing Side Channels Peter W. Deutsch, Yuheng Yang, Thomas Bourgeat, Jules Drean, Joel S. Emer, and Mengjia Yan (Massachusetts Institute of Technology, USA; NVIDIA, USA) This paper studies the mitigation of memory timing side channels, where attackers utilize contention within DRAM controllers to infer a victim’s secrets. Already practical, this class of channels poses an important challenge to secure computing in shared memory environments. Existing state-of-the-art memory timing side channel mitigations have several key performance and security limitations. Prior schemes require onerous static bandwidth partitioning, extensive profiling phases, or simply fail to protect against attacks which exploit fine-grained timing and bank information. We present DAGguise, a defense mechanism which fully protects against memory timing side channels while allowing for dynamic traffic contention in order to achieve good performance. DAGguise utilizes a novel abstract memory access representation, the Directed Acyclic Request Graph (rDAG for short), to model memory access patterns which experience contention. DAGguise shapes a victim’s memory access patterns according to a publicly known rDAG obtained through a lightweight profiling stage, completely eliminating information leakage. We formally verify the security of DAGguise, proving that it maintains strong security guarantees. Moreover, by allowing dynamic traffic contention, DAGguise achieves a 12% overall system speedup relative to Fixed Service, which is the state-of-the-art mitigation mechanism, with up to a 20% relative speedup for co-located applications which do not require protection. We further claim that the principles of DAGguise can be generalized to protect against other types of scheduler-based timing side channels, such as those targeting on-chip networks, or functional units in SMT cores. @InProceedings{ASPLOS22p329, author = {Peter W. Deutsch and Yuheng Yang and Thomas Bourgeat and Jules Drean and Joel S. Emer and Mengjia Yan}, title = {DAGguise: Mitigating Memory Timing Side Channels}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {329--343}, doi = {10.1145/3503222.3507747}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Eran, Haggai	ASPLOS '22: "FlexDriver: A Network Driver ..." FlexDriver: A Network Driver for Your Accelerator Haggai Eran, Maxim Fudim, Gabi Malka, Gal Shalom, Noam Cohen, Amit Hermony, Dotan Levi, Liran Liss, and Mark Silberstein (NVIDIA, Israel; Technion, Israel) We propose a new system design for connecting hardware and FPGA accelerators to the network, allowing the accelerator to directly control commodity Network Interface Cards (NICs) without using the CPU. This enables us to solve the key challenge of leveraging existing NIC hardware offloads such as virtualization, tunneling, and RDMA for accelerator networking. Our approach supports a diverse set of use cases, from direct network access for disaggregated accelerators to inline-acceleration of the network stack, all without the complex networking logic in the accelerator. To demonstrate the feasibility of this approach, we build FlexDriver (FLD), an on-accelerator hardware module that implements a NIC data-plane driver. Our main technical contribution is a mechanism that compresses the NIC control structures by two orders of magnitude, allowing FLD to achieve high networking scalability with low die area cost and no bandwidth interference with the accelerator logic. The prototype for NVIDIA Innova-2 FPGA SmartNICs showcases our design’s utility for three different accelerators: a disaggregated LTE cipher, an IP-defragmentation inline accelerator, and an IoT cryptographic-token authentication offload. These accelerators reach 25 Gbps line rate and leverage the NIC for RDMA processing, VXLAN tunneling, and traffic shaping without CPU involvement. @InProceedings{ASPLOS22p1115, author = {Haggai Eran and Maxim Fudim and Gabi Malka and Gal Shalom and Noam Cohen and Amit Hermony and Dotan Levi and Liran Liss and Mark Silberstein}, title = {FlexDriver: A Network Driver for Your Accelerator}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1115--1129}, doi = {10.1145/3503222.3507776}, year = {2022}, } Publisher's Version
	Erdős, Márton	ASPLOS '22: "MineSweeper: A “Clean Sweep” ..." MineSweeper: A “Clean Sweep” for Drop-In Use-after-Free Prevention Márton Erdős, Sam Ainsworth, and Timothy M. Jones (University of Cambridge, UK; University of Edinburgh, UK) Low-level languages, which require manual memory management from the programmer, remain in wide use for performance-critical applications. Memory-safety bugs are common, and now a major source of exploits. In particular, a use-after-free bug occurs when an object is erroneously deallocated, whilst pointers to it remain active in memory, and those (dangling) pointers are later used to access the object. An attacker can reallocate the memory area backing an erroneously freed object, then overwrite its contents, injecting carefully chosen data into the host program, thus altering its execution and achieving privilege escalation. We present MineSweeper, a system to mitigate use-after-free vulnerabilities by retaining freed allocations in a quarantine, until no pointers to them remain in program memory, thus preventing their reallocation until it is safe. MineSweeper performs efficient linear sweeps of memory to identify quarantined items that have no dangling pointers to them, and thus can be safely reallocated. This allows MineSweeper to be significantly more efficient than previous transitive marking procedure techniques. MineSweeper, attached to JeMalloc, improves security at an acceptable overhead in memory footprint (11.1% on average) and an execution-time cost of only 5.4% (geometric mean for SPEC CPU2006), with 9.6% additional threaded CPU usage. These figures considerably improve on the state-of-the-art for non-probabilistic drop-in temporal-safety systems, and make MineSweeper the only such scheme suitable for deployment in real-world production environments. @InProceedings{ASPLOS22p212, author = {Márton Erdős and Sam Ainsworth and Timothy M. Jones}, title = {MineSweeper: A “Clean Sweep” for Drop-In Use-after-Free Prevention}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {212--225}, doi = {10.1145/3503222.3507712}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Feng, Yu	ASPLOS '22: "Tree Traversal Synthesis Using ..." Tree Traversal Synthesis Using Domain-Specific Symbolic Compilation Yanju Chen, Junrui Liu, Yu Feng, and Rastislav Bodik (University of California at Santa Barbara, USA; University of Washington, USA) Efficient computation on tree data structures is important in compilers, numeric computations, and web browser layout engines. Efficiency is achieved by statically scheduling the computation into a small number of tree traversals and by performing the traversals in parallel when possible. Manual design of such traversals leads to bugs, as observed in web browsers. Automatic schedulers avoid these bugs but they currently cannot explore a space of legal traversals, which prevents exploring the trade-offs between parallelism and minimizing the number of traversals. We describe Hecate, a synthesizer of tree traversals that can produce both serial and parallel traversals. A key feature is that the synthesizer is extensible by the programmer who can define a template for new kinds of traversals. Hecate is constructed as a solver-aided domain-specific language, meaning that the synthesizer is generated automatically by translating the tree traversal DSL to an SMT solver that synthesizes the traversals. We improve on the general-purpose solver-aided architecture with a scheduling-specific symbolic evaluation that maintains the engineering advantages solver-aided design but generates efficient ILP encoding that is much more efficient to solve than SMT constraints. On the set of Grafter problems, Hecate synthesizes traversals that trade off traversal fusion to exploit parallelism. Additionally, Hecate allows defining a tree data structure with an arbitrary number of children. Together, parallelism and data structure improvements accelerate the computation 2× on a tree rendering problem. Finally, Hecate’s domain-specific symbolic compilation accelerates synthesis 3× compared to the general-purpose compilation to an SMT solver; when scheduling a CSS engine traversal, this ILP-based synthesis executes orders of magnitude faster. @InProceedings{ASPLOS22p1030, author = {Yanju Chen and Junrui Liu and Yu Feng and Rastislav Bodik}, title = {Tree Traversal Synthesis Using Domain-Specific Symbolic Compilation}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1030--1042}, doi = {10.1145/3503222.3507751}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional
	Fetzer, Christof	ASPLOS '22: "Revizor: Testing Black-Box ..." Revizor: Testing Black-Box CPUs against Speculation Contracts Oleksii Oleksenko, Christof Fetzer, Boris Köpf, and Mark Silberstein (TU Dresden, Germany; Microsoft Research, UK; Technion, Israel) Speculative vulnerabilities such as Spectre and Meltdown expose speculative execution state that can be exploited to leak information across security domains via side-channels. Such vulnerabilities often stay undetected for a long time as we lack the tools for systematic testing of CPUs to find them. In this paper, we propose an approach to automatically detect microarchitectural information leakage in commercial black-box CPUs. We build on speculation contracts, which we employ to specify the permitted side effects of program execution on the CPU's microarchitectural state. We propose a Model-based Relational Testing (MRT) technique to empirically assess the CPU compliance with these specifications. We implement MRT in a testing framework called Revizor, and showcase its effectiveness on real Intel x86 CPUs. Revizor automatically detects violations of a rich set of contracts, or indicates their absence. A highlight of our findings is that Revizor managed to automatically surface Spectre, MDS, and LVI, as well as several previously unknown variants. @InProceedings{ASPLOS22p226, author = {Oleksii Oleksenko and Christof Fetzer and Boris Köpf and Mark Silberstein}, title = {Revizor: Testing Black-Box CPUs against Speculation Contracts}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {226--239}, doi = {10.1145/3503222.3507729}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Firtina, Can	ASPLOS '22: "GenStore: A High-Performance ..." GenStore: A High-Performance In-Storage Processing System for Genome Sequence Analysis Nika Mansouri Ghiasi, Jisung Park, Harun Mustafa, Jeremie Kim, Ataberk Olgun, Arvid Gollwitzer, Damla Senol Cali, Can Firtina, Haiyu Mao, Nour Almadhoun Alserr, Rachata Ausavarungnirun, Nandita Vijaykumar, Mohammed Alser, and Onur Mutlu (ETH Zurich, Switzerland; Bionano Genomics, USA; King Mongkut's University of Technology North Bangkok, Thailand; University of Toronto, Canada) Read mapping is a fundamental step in many genomics applications. It is used to identify potential matches and differences between fragments (called reads) of a sequenced genome and an already known genome (called a reference genome). Read mapping is costly because it needs to perform approximate string matching (ASM) on large amounts of data. To address the computational challenges in genome analysis, many prior works propose various approaches such as accurate filters that select the reads within a dataset of genomic reads (called a read set) that must undergo expensive computation, efficient heuristics, and hardware acceleration. While effective at reducing the amount of expensive computation, all such approaches still require the costly movement of a large amount of data from storage to the rest of the system, which can significantly lower the end-to-end performance of read mapping in conventional and emerging genomics systems. We propose GenStore, the first in-storage processing system designed for genome sequence analysis that greatly reduces both data movement and computational overheads of genome sequence analysis by exploiting low-cost and accurate in-storage filters. GenStore leverages hardware/software co-design to address the challenges of in-storage processing, supporting reads with 1) different properties such as read lengths and error rates, which highly depend on the sequencing technology, and 2) different degrees of genetic variation compared to the reference genome, which highly depends on the genomes that are being compared. Through rigorous analysis of read mapping processes of reads with different properties and degrees of genetic variation, we meticulously design low-cost hardware accelerators and data/computation flows inside a NAND flash-based solid-state drive (SSD). Our evaluation using a wide range of real genomic datasets shows that GenStore, when implemented in three modern NAND flash-based SSDs, significantly improves the read mapping performance of state-of-the-art software (hardware) baselines by 2.07-6.05× (1.52-3.32×) for read sets with high similarity to the reference genome and 1.45-33.63× (2.70-19.2×) for read sets with low similarity to the reference genome. @InProceedings{ASPLOS22p635, author = {Nika Mansouri Ghiasi and Jisung Park and Harun Mustafa and Jeremie Kim and Ataberk Olgun and Arvid Gollwitzer and Damla Senol Cali and Can Firtina and Haiyu Mao and Nour Almadhoun Alserr and Rachata Ausavarungnirun and Nandita Vijaykumar and Mohammed Alser and Onur Mutlu}, title = {GenStore: A High-Performance In-Storage Processing System for Genome Sequence Analysis}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {635--654}, doi = {10.1145/3503222.3507702}, year = {2022}, } Publisher's Version
	Fu, Kaihua	ASPLOS '22: "Astraea: Towards QoS-Aware ..." Astraea: Towards QoS-Aware and Resource-Efficient Multi-stage GPU Services Wei Zhang, Quan Chen, Kaihua Fu, Ningxin Zheng, Zhiyi Huang, Jingwen Leng, and Minyi Guo (Shanghai Jiao Tong University, China; Microsoft Research, China; University of Otago, New Zealand) Multi-stage user-facing applications on GPUs are widely-used nowa- days, and are often implemented to be microservices. Prior re- search works are not applicable to ensuring QoS of GPU-based microservices due to the different communication patterns and shared resource contentions. We propose Astraea to manage GPU microservices considering the above factors. In Astraea, a microser- vice deployment policy is used to maximize the supported peak service load while ensuring the required QoS. To adaptively switch the communication methods between microservices according to different deployments, we propose an auto-scaling GPU communi- cation framework. The framework automatically scales based on the currently used hardware topology and microservice location, and adopts global memory-based techniques to reduce intra-GPU communication. Astraea increases the supported peak load by up to 82.3% while achieving the desired 99%-ile latency target compared with state-of-the-art solutions. @InProceedings{ASPLOS22p570, author = {Wei Zhang and Quan Chen and Kaihua Fu and Ningxin Zheng and Zhiyi Huang and Jingwen Leng and Minyi Guo}, title = {Astraea: Towards QoS-Aware and Resource-Efficient Multi-stage GPU Services}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {570--582}, doi = {10.1145/3503222.3507721}, year = {2022}, } Publisher's Version
	Fudim, Maxim	ASPLOS '22: "FlexDriver: A Network Driver ..." FlexDriver: A Network Driver for Your Accelerator Haggai Eran, Maxim Fudim, Gabi Malka, Gal Shalom, Noam Cohen, Amit Hermony, Dotan Levi, Liran Liss, and Mark Silberstein (NVIDIA, Israel; Technion, Israel) We propose a new system design for connecting hardware and FPGA accelerators to the network, allowing the accelerator to directly control commodity Network Interface Cards (NICs) without using the CPU. This enables us to solve the key challenge of leveraging existing NIC hardware offloads such as virtualization, tunneling, and RDMA for accelerator networking. Our approach supports a diverse set of use cases, from direct network access for disaggregated accelerators to inline-acceleration of the network stack, all without the complex networking logic in the accelerator. To demonstrate the feasibility of this approach, we build FlexDriver (FLD), an on-accelerator hardware module that implements a NIC data-plane driver. Our main technical contribution is a mechanism that compresses the NIC control structures by two orders of magnitude, allowing FLD to achieve high networking scalability with low die area cost and no bandwidth interference with the accelerator logic. The prototype for NVIDIA Innova-2 FPGA SmartNICs showcases our design’s utility for three different accelerators: a disaggregated LTE cipher, an IP-defragmentation inline accelerator, and an IoT cryptographic-token authentication offload. These accelerators reach 25 Gbps line rate and leverage the NIC for RDMA processing, VXLAN tunneling, and traffic shaping without CPU involvement. @InProceedings{ASPLOS22p1115, author = {Haggai Eran and Maxim Fudim and Gabi Malka and Gal Shalom and Noam Cohen and Amit Hermony and Dotan Levi and Liran Liss and Mark Silberstein}, title = {FlexDriver: A Network Driver for Your Accelerator}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1115--1129}, doi = {10.1145/3503222.3507776}, year = {2022}, } Publisher's Version
	Fuerst, Alexander	ASPLOS '22: "Memory-Harvesting VMs in Cloud ..." Memory-Harvesting VMs in Cloud Platforms Alexander Fuerst, Stanko Novaković, Íñigo Goiri, Gohar Irfan Chaudhry, Prateek Sharma, Kapil Arya, Kevin Broas, Eugene Bak, Mehmet Iyigun, and Ricardo Bianchini (Indiana University, USA; Microsoft Research, USA; Microsoft Azure, USA) loud platforms monetize their spare capacity by renting “Spot” virtual machines (VMs) that can be evicted in favor of higher-priority VMs. Recent work has shown that resource-harvesting VMs are more effective at exploiting spare capacity than Spot VMs, while also reducing the number of evictions. However, the prior work focused on harvesting CPU cores while keeping memory size fixed. This wastes a substantial monetization opportunity and may even limit the ability of harvesting VMs to leverage spare cores. Thus, in this paper, we explore memory harvesting and its challenges in real cloud platforms, namely its impact on VM creation time, NUMA spanning, and page fragmentation. We start by characterizing the amount and dynamics of the spare memory in Azure. We then design and implement memory-harvesting VMs (MHVMs), introducing new techniques for memory buffering, batching, and pre-reclamation. To demonstrate the use of MHVMs, we also extend a popular cluster scheduling framework (Hadoop) and a FaaS platform to adapt to them. Our main results show that (1) there is plenty of scope for memory harvesting in real platforms; (2) MHVMs are effective at mitigating the negative impacts of harvesting; and (3) our extensions of Hadoop and FaaS successfully hide the MHVMs’ varying memory size from the users’ data-processing jobs and functions. We conclude that memory harvesting has great potential for practical deployment and users can save up to 93% of their costs when running workloads on MHVMs. @InProceedings{ASPLOS22p583, author = {Alexander Fuerst and Stanko Novaković and Íñigo Goiri and Gohar Irfan Chaudhry and Prateek Sharma and Kapil Arya and Kevin Broas and Eugene Bak and Mehmet Iyigun and Ricardo Bianchini}, title = {Memory-Harvesting VMs in Cloud Platforms}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {583--594}, doi = {10.1145/3503222.3507725}, year = {2022}, } Publisher's Version
	Gao, Mingyu	ASPLOS '22: "ShEF: Shielded Enclaves for ..." ShEF: Shielded Enclaves for Cloud FPGAs Mark Zhao, Mingyu Gao, and Christos Kozyrakis (Stanford University, USA; Tsinghua University, China) FPGAs are now used in public clouds to accelerate a wide range of applications, including many that operate on sensitive data such as financial and medical records. We present ShEF, a trusted execution environment (TEE) for cloud-based reconfigurable accelerators. ShEF is independent from CPU-based TEEs and allows secure execution under a threat model where the adversary can control all software running on the CPU connected to the FPGA, has physical access to the FPGA, and can compromise the FPGA interface logic of the cloud provider. ShEF provides a secure boot and remote attestation process that relies solely on existing FPGA mechanisms for root of trust. It also includes a Shield component that provides secure access to data while the accelerator is in use. The Shield is highly customizable and extensible, allowing users to craft a bespoke security solution that fits their accelerator's memory access patterns, bandwidth, and security requirements at minimum performance and area overheads. We describe a prototype implementation of ShEF for existing cloud FPGAs, map ShEF to a performant and secure storage application, and measure the performance benefits of customizable security using five additional accelerators. @InProceedings{ASPLOS22p1070, author = {Mark Zhao and Mingyu Gao and Christos Kozyrakis}, title = {ShEF: Shielded Enclaves for Cloud FPGAs}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1070--1085}, doi = {10.1145/3503222.3507733}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional ASPLOS '22: "FINGERS: Exploiting Fine-Grained ..." FINGERS: Exploiting Fine-Grained Parallelism in Graph Mining Accelerators Qihang Chen, Boyu Tian, and Mingyu Gao (Tsinghua University, China) Graph mining is an emerging application of high importance and also with high complexity, thus requiring efficient hardware acceleration. Current accelerator designs only utilize coarse-grained parallelism, leaving large room for further optimizations. Our key insight is to fully exploit fine-grained parallelism to overcome the existing issues of hardware underutilization, inefficient resource provision, and limited single-thread performance under imbalanced loads. Targeting pattern-aware graph mining algorithms, we first comprehensively identify and analyze the abundant fine-grained parallelism at the branch, set, and segment levels during search tree exploration and set operations. We then propose a novel graph mining accelerator, FINGERS, which effectively exploits these multiple levels of fine-grained parallelism to achieve significant performance improvements. FINGERS mainly enhances the design of each single processing element with parallel compute units for set operations, and efficient techniques for task scheduling, load balancing, and data aggregation. FINGERS outperforms the state-of-the-art design by 2.8× on average and up to 8.9× with the same chip area. We also demonstrate that different patterns and different graphs exhibit drastically different parallelism opportunities, justifying the necessity of exploiting all levels of fine-grained parallelism in FINGERS. @InProceedings{ASPLOS22p43, author = {Qihang Chen and Boyu Tian and Mingyu Gao}, title = {FINGERS: Exploiting Fine-Grained Parallelism in Graph Mining Accelerators}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {43--55}, doi = {10.1145/3503222.3507730}, year = {2022}, } Publisher's Version
	Gao, Wei	ASPLOS '22: "Eavesdropping User Credentials ..." Eavesdropping User Credentials via GPU Side Channels on Smartphones Boyuan Yang, Ruirong Chen, Kai Huang, Jun Yang, and Wei Gao (University of Pittsburgh, USA) Graphics Processing Unit (GPU) on smartphones is an effective target for hardware attacks. In this paper, we present a new side channel attack on mobile GPUs of Android smartphones, allowing an unprivileged attacker to eavesdrop the user's credentials, such as login usernames and passwords, from their inputs through on-screen keyboard. Our attack targets on Qualcomm Adreno GPUs and investigate the amount of GPU overdraw when rendering the popups of user's key presses of inputs. Such GPU overdraw caused by each key press corresponds to unique variations of selected GPU performance counters, from which these key presses can be accurately inferred. Experiment results from practical use on multiple models of Android smartphones show that our attack can correctly infer more than 80% of user's credential inputs, but incur negligible amounts of computing overhead and network traffic on the victim device. To counter this attack, this paper suggests mitigations of access control on GPU performance counters, or applying obfuscations on the values of GPU performance counters. @InProceedings{ASPLOS22p285, author = {Boyuan Yang and Ruirong Chen and Kai Huang and Jun Yang and Wei Gao}, title = {Eavesdropping User Credentials via GPU Side Channels on Smartphones}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {285--299}, doi = {10.1145/3503222.3507757}, year = {2022}, } Publisher's Version Published Artifact Video Artifacts Available Artifacts Functional
	Gaur, Ishan	ASPLOS '22: "Taurus: A Data Plane Architecture ..." Taurus: A Data Plane Architecture for Per-Packet ML Tushar Swamy, Alexander Rucker, Muhammad Shahbaz, Ishan Gaur, and Kunle Olukotun (Stanford University, USA; Purdue University, USA) Emerging applications---cloud computing, the internet of things, and augmented/virtual reality---demand responsive, secure, and scalable datacenter networks. These networks currently implement simple, per-packet, data-plane heuristics (e.g., ECMP and sketches) under a slow, millisecond-latency control plane that runs data-driven performance and security policies. However, to meet applications' service-level objectives (SLOs) in a modern data center, networks must bridge the gap between line-rate, per-packet execution and complex decision making. In this work, we present the design and implementation of Taurus, a data plane for line-rate inference. Taurus adds custom hardware based on a flexible, parallel-patterns (MapReduce) abstraction to programmable network devices, such as switches and NICs; this new hardware uses pipelined SIMD parallelism to enable per-packet MapReduce operations (e.g., inference). Our evaluation of a Taurus switch ASIC---supporting several real-world models---shows that Taurus operates orders of magnitude faster than a server-based control plane while increasing area by 3.8% and latency for line-rate ML models by up to 221 ns. Furthermore, our Taurus FPGA prototype achieves full model accuracy and detects two orders of magnitude more events than a state-of-the-art control-plane anomaly-detection system. @InProceedings{ASPLOS22p1099, author = {Tushar Swamy and Alexander Rucker and Muhammad Shahbaz and Ishan Gaur and Kunle Olukotun}, title = {Taurus: A Data Plane Architecture for Per-Packet ML}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1099--1114}, doi = {10.1145/3503222.3507726}, year = {2022}, } Publisher's Version Published Artifact Info Artifacts Available
	Ghosh, Souradip	ASPLOS '22: "CARAT CAKE: Replacing Paging ..." CARAT CAKE: Replacing Paging via Compiler/Kernel Cooperation Brian Suchy, Souradip Ghosh, Drew Kersnar, Siyuan Chai, Zhen Huang, Aaron Nelson, Michael Cuevas, Alex Bernat, Gaurav Chaudhary, Nikos Hardavellas, Simone Campanoni, and Peter Dinda (Northwestern University, USA) Virtual memory, specifically paging, is undergoing significant innovation due to being challenged by new demands from modern workloads. Recent work has demonstrated an alternative software only design that can result in simplified hardware requirements, even supporting purely physical addressing. While we have made the case for this Compiler- And Runtime-based Address Translation (CARAT) concept, its evaluation was based on a user-level prototype. We now report on incorporating CARAT into a kernel, forming Compiler- And Runtime-based Address Translation for CollAborative Kernel Environments (CARAT CAKE). In our implementation, a Linux-compatible x64 process abstraction can be based either on CARAT CAKE, or on a sophisticated paging implementation. Implementing CARAT CAKE involves kernel changes and compiler optimizations/transformations that must work on all code in the system, including kernel code. We evaluate CARAT CAKE in comparison with paging and find that CARAT CAKE is able to achieve the functionality of paging (protection, mapping, and movement properties) with minimal overhead. In turn, CARAT CAKE allows significant new benefits for systems including energy savings, larger L1 caches, and arbitrary granularity memory management. @InProceedings{ASPLOS22p98, author = {Brian Suchy and Souradip Ghosh and Drew Kersnar and Siyuan Chai and Zhen Huang and Aaron Nelson and Michael Cuevas and Alex Bernat and Gaurav Chaudhary and Nikos Hardavellas and Simone Campanoni and Peter Dinda}, title = {CARAT CAKE: Replacing Paging via Compiler/Kernel Cooperation}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {98--114}, doi = {10.1145/3503222.3507771}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available
	Giardino, Michael	ASPLOS '22: "Enzian: An Open, General, ..." Enzian: An Open, General, CPU/FPGA Platform for Systems Software Research David Cock, Abishek Ramdas, Daniel Schwyn, Michael Giardino, Adam Turowski, Zhenhao He, Nora Hossle, Dario Korolija, Melissa Licciardello, Kristina Martsenko, Reto Achermann, Gustavo Alonso, and Timothy Roscoe (ETH Zurich, Switzerland; University of British Columbia, Canada) Hybrid computing platforms, comprising CPU cores and FPGA logic, are increasingly used for accelerating data-intensive workloads in cloud deployments, and are a growing topic of interest in systems research. However, from a research perspective, existing hardware platforms are limited: they are often optimized for concrete, narrow use-cases and, therefore lack the flexibility needed to explore other applications and configurations. We show that a research group can design and build a more general, open, and affordable hardware platform for hybrid systems research. The platform, Enzian, is capable of duplicating the functionality of existing CPU/FPGA systems with comparable performance but in an open, flexible system. It couples a large FPGA with a server-class CPU in an asymmetric cache-coherent NUMA system. Enzian also enables research not possible with existing hybrid platforms, through explicit access to coherence messages, extensive thermal and power instrumentation, and an open, programmable baseboard management processor. Enzian is already being used in multiple projects, is open source (both hardware and software), and available for remote use. We present the design principles of Enzian, the challenges in building it, and evaluate it with a range of existing research use-cases alongside other, more specialized platforms, as well as demonstrating research not possible on existing platforms. @InProceedings{ASPLOS22p434, author = {David Cock and Abishek Ramdas and Daniel Schwyn and Michael Giardino and Adam Turowski and Zhenhao He and Nora Hossle and Dario Korolija and Melissa Licciardello and Kristina Martsenko and Reto Achermann and Gustavo Alonso and Timothy Roscoe}, title = {Enzian: An Open, General, CPU/FPGA Platform for Systems Software Research}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {434--451}, doi = {10.1145/3503222.3507742}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Goiri, Íñigo	ASPLOS '22: "Memory-Harvesting VMs in Cloud ..." Memory-Harvesting VMs in Cloud Platforms Alexander Fuerst, Stanko Novaković, Íñigo Goiri, Gohar Irfan Chaudhry, Prateek Sharma, Kapil Arya, Kevin Broas, Eugene Bak, Mehmet Iyigun, and Ricardo Bianchini (Indiana University, USA; Microsoft Research, USA; Microsoft Azure, USA) loud platforms monetize their spare capacity by renting “Spot” virtual machines (VMs) that can be evicted in favor of higher-priority VMs. Recent work has shown that resource-harvesting VMs are more effective at exploiting spare capacity than Spot VMs, while also reducing the number of evictions. However, the prior work focused on harvesting CPU cores while keeping memory size fixed. This wastes a substantial monetization opportunity and may even limit the ability of harvesting VMs to leverage spare cores. Thus, in this paper, we explore memory harvesting and its challenges in real cloud platforms, namely its impact on VM creation time, NUMA spanning, and page fragmentation. We start by characterizing the amount and dynamics of the spare memory in Azure. We then design and implement memory-harvesting VMs (MHVMs), introducing new techniques for memory buffering, batching, and pre-reclamation. To demonstrate the use of MHVMs, we also extend a popular cluster scheduling framework (Hadoop) and a FaaS platform to adapt to them. Our main results show that (1) there is plenty of scope for memory harvesting in real platforms; (2) MHVMs are effective at mitigating the negative impacts of harvesting; and (3) our extensions of Hadoop and FaaS successfully hide the MHVMs’ varying memory size from the users’ data-processing jobs and functions. We conclude that memory harvesting has great potential for practical deployment and users can save up to 93% of their costs when running workloads on MHVMs. @InProceedings{ASPLOS22p583, author = {Alexander Fuerst and Stanko Novaković and Íñigo Goiri and Gohar Irfan Chaudhry and Prateek Sharma and Kapil Arya and Kevin Broas and Eugene Bak and Mehmet Iyigun and Ricardo Bianchini}, title = {Memory-Harvesting VMs in Cloud Platforms}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {583--594}, doi = {10.1145/3503222.3507725}, year = {2022}, } Publisher's Version
	Goldie, Anna	ASPLOS '22: "A Full-Stack Search Technique ..." A Full-Stack Search Technique for Domain Optimized Deep Learning Accelerators Dan Zhang, Safeen Huda, Ebrahim Songhori, Kartik Prabhu, Quoc Le, Anna Goldie, and Azalia Mirhoseini (Google Brain, USA; Google, USA; Stanford University, USA) The rapidly-changing deep learning landscape presents a unique opportunity for building inference accelerators optimized for specific datacenter-scale workloads. We propose Full-stack Accelerator Search Technique (FAST), a hardware accelerator search framework that defines a broad optimization environment covering key design decisions within the hardware-software stack, including hardware datapath, software scheduling, and compiler passes such as operation fusion and tensor padding. In this paper, we analyze bottlenecks in state-of-the-art vision and natural language processing (NLP) models, including EfficientNet and BERT, and use FAST to design accelerators capable of addressing these bottlenecks. FAST-generated accelerators optimized for single workloads improve Perf/TDP by 3.7× on average across all benchmarks compared to TPU-v3. A FAST-generated accelerator optimized for serving a suite of workloads improves Perf/TDP by 2.4× on average compared to TPU-v3. Our return on investment analysis shows that FAST-generated accelerators can potentially be practical for moderate-sized datacenter deployments. @InProceedings{ASPLOS22p27, author = {Dan Zhang and Safeen Huda and Ebrahim Songhori and Kartik Prabhu and Quoc Le and Anna Goldie and Azalia Mirhoseini}, title = {A Full-Stack Search Technique for Domain Optimized Deep Learning Accelerators}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {27--42}, doi = {10.1145/3503222.3507767}, year = {2022}, } Publisher's Version
	Goldsmith, Matthew	ASPLOS '22: "PLD: Fast FPGA Compilation ..." PLD: Fast FPGA Compilation to Make Reconfigurable Acceleration Compatible with Modern Incremental Refinement Software Development Yuanlong Xiao, Eric Micallef, Andrew Butt, Matthew Hofmann, Marc Alston, Matthew Goldsmith, Andrew Merczynski-Hait, and André DeHon (University of Pennsylvania, USA) FPGA-based accelerators are demonstrating significant absolute performance and energy efficiency compared with general-purpose CPUs. While FPGA computations can now be described in standard, programming languages, like C, development for FPGAs accelerators remains tedious and inaccessible to modern software engineers. Slow compiles (potentially taking tens of hours) inhibit the rapid, incremental refinement of designs that is the hallmark of modern software engineering. To address this issue, we introduce separate compilation and linkage into the FPGA design flow, providing faster design turns more familiar to software development. To realize this flow, we provide abstractions, compiler options, and compiler flow that allow the same C source code to be compiled to processor cores in seconds and to FPGA regions in minutes, providing the missing -O0 and -O1 options familiar in software development. This raises the FPGA programming level and standardizes the programming experience, bringing FPGA-based accelerators into a more familiar software platform ecosystem for software engineers. @InProceedings{ASPLOS22p933, author = {Yuanlong Xiao and Eric Micallef and Andrew Butt and Matthew Hofmann and Marc Alston and Matthew Goldsmith and Andrew Merczynski-Hait and André DeHon}, title = {PLD: Fast FPGA Compilation to Make Reconfigurable Acceleration Compatible with Modern Incremental Refinement Software Development}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {933--945}, doi = {10.1145/3503222.3507740}, year = {2022}, } Publisher's Version
	Gollwitzer, Arvid	ASPLOS '22: "GenStore: A High-Performance ..." GenStore: A High-Performance In-Storage Processing System for Genome Sequence Analysis Nika Mansouri Ghiasi, Jisung Park, Harun Mustafa, Jeremie Kim, Ataberk Olgun, Arvid Gollwitzer, Damla Senol Cali, Can Firtina, Haiyu Mao, Nour Almadhoun Alserr, Rachata Ausavarungnirun, Nandita Vijaykumar, Mohammed Alser, and Onur Mutlu (ETH Zurich, Switzerland; Bionano Genomics, USA; King Mongkut's University of Technology North Bangkok, Thailand; University of Toronto, Canada) Read mapping is a fundamental step in many genomics applications. It is used to identify potential matches and differences between fragments (called reads) of a sequenced genome and an already known genome (called a reference genome). Read mapping is costly because it needs to perform approximate string matching (ASM) on large amounts of data. To address the computational challenges in genome analysis, many prior works propose various approaches such as accurate filters that select the reads within a dataset of genomic reads (called a read set) that must undergo expensive computation, efficient heuristics, and hardware acceleration. While effective at reducing the amount of expensive computation, all such approaches still require the costly movement of a large amount of data from storage to the rest of the system, which can significantly lower the end-to-end performance of read mapping in conventional and emerging genomics systems. We propose GenStore, the first in-storage processing system designed for genome sequence analysis that greatly reduces both data movement and computational overheads of genome sequence analysis by exploiting low-cost and accurate in-storage filters. GenStore leverages hardware/software co-design to address the challenges of in-storage processing, supporting reads with 1) different properties such as read lengths and error rates, which highly depend on the sequencing technology, and 2) different degrees of genetic variation compared to the reference genome, which highly depends on the genomes that are being compared. Through rigorous analysis of read mapping processes of reads with different properties and degrees of genetic variation, we meticulously design low-cost hardware accelerators and data/computation flows inside a NAND flash-based solid-state drive (SSD). Our evaluation using a wide range of real genomic datasets shows that GenStore, when implemented in three modern NAND flash-based SSDs, significantly improves the read mapping performance of state-of-the-art software (hardware) baselines by 2.07-6.05× (1.52-3.32×) for read sets with high similarity to the reference genome and 1.45-33.63× (2.70-19.2×) for read sets with low similarity to the reference genome. @InProceedings{ASPLOS22p635, author = {Nika Mansouri Ghiasi and Jisung Park and Harun Mustafa and Jeremie Kim and Ataberk Olgun and Arvid Gollwitzer and Damla Senol Cali and Can Firtina and Haiyu Mao and Nour Almadhoun Alserr and Rachata Ausavarungnirun and Nandita Vijaykumar and Mohammed Alser and Onur Mutlu}, title = {GenStore: A High-Performance In-Storage Processing System for Genome Sequence Analysis}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {635--654}, doi = {10.1145/3503222.3507702}, year = {2022}, } Publisher's Version
	Gonzalez-Guerrero, Patricia	ASPLOS '22: "Temporal and SFQ Pulse-Streams ..." Temporal and SFQ Pulse-Streams Encoding for Area-Efficient Superconducting Accelerators Patricia Gonzalez-Guerrero, Meriam Gay Bautista, Darren Lyles, and George Michelogiannakis (Lawrence Berkeley National Laboratory, USA) Superconducting technology is a prime candidate for the future of computing. However, current superconducting prototypes are limited to small-scale examples due to stringent area constraints and complex architectures inspired from voltage-level encoding in CMOS; this is at odds with the ps-wide Single Quantum Flux (SFQ) pulses used in superconductors to carry information. In this work, we propose a wave-pipelined Unary SFQ (U-SFQ) architecture that leverages the advantages of two data representations: pulse-streams and Race Logic (RL). We introduce novel building blocks such as multipliers, adders, and memory cells, which leverage the natural properties of SFQ pulses to mitigate area constraints. We then design and simulate three popular hardware accelerators: i) a Processing Element (PE), typically used in spatial architectures; ii) A dot-product-unit (DPU), one of the most popular accelerators in artificial neural networks and digital signal processing (DSP); and iii) A Finite Impulse Response (FIR) filter, a popular and computationally demanding DSP accelerator. The proposed U-SFQ building blocks require up to 200× fewer JJs compared to their SFQ binary counterparts, exposing an area-delay trade-off. This work mitigates the stringent area constraints of superconducting technology. @InProceedings{ASPLOS22p963, author = {Patricia Gonzalez-Guerrero and Meriam Gay Bautista and Darren Lyles and George Michelogiannakis}, title = {Temporal and SFQ Pulse-Streams Encoding for Area-Efficient Superconducting Accelerators}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {963--976}, doi = {10.1145/3503222.3507765}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available
	Gorjiara, Hamed	ASPLOS '22: "Yashme: Detecting Persistency ..." Yashme: Detecting Persistency Races Hamed Gorjiara, Guoqing Harry Xu, and Brian Demsky (University of California at Irvine, USA; University of California at Los Angeles, USA) Persistent memory (PM) or Non-Volatile Random-Access Memory (NVRAM) hardware such as Intel’s Optane memory product promises to transform how programs store and manipulate information. Ensuring that persistent memory programs are crash consistent is a major challenge. We present a novel class of crash consistency bugs for persistent memory programs, which we call persistency races. Persistency races can cause non-atomic stores to be made partially persistent. Persistency races arise due to the interaction of standard compiler optimizations with persistent memory semantics. We present Yashme, the first detector for persistency races. A major challenge is that in order to detect persistency races, the execution must crash in a very narrow window between a store with a persistency race and its corresponding cache flush operation, making it challenging for naïve techniques to be effective. Yashme overcomes this challenge with a novel technique for detecting races in executions that are prefixes of the pre-crash execution. This technique enables Yashme to effectively find persistency races even if the injected crashes do not fall into that window. We have evaluated Yashme on a range of persistent memory benchmarks and have found 26 real persistency races that have never been reported before. @InProceedings{ASPLOS22p830, author = {Hamed Gorjiara and Guoqing Harry Xu and Brian Demsky}, title = {Yashme: Detecting Persistency Races}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {830--845}, doi = {10.1145/3503222.3507766}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Grosser, Tobias	ASPLOS '22: "Understanding and Exploiting ..." Understanding and Exploiting Optimal Function Inlining Theodoros Theodoridis, Tobias Grosser, and Zhendong Su (ETH Zurich, Switzerland; University of Edinburgh, UK) Inlining is a core transformation in optimizing compilers. It replaces a function call (call site) with the body of the called function (callee). It helps reduce function call overhead and binary size, and more importantly, enables other optimizations. The problem of inlining has been extensively studied, but it is far from being solved; predicting which inlining decisions are beneficial is nontrivial due to interactions with the rest of the compiler pipeline. Previous work has mainly focused on designing heuristics for better inlining decisions and has not investigated optimal inlining, i.e., exhaustively finding the optimal inlining decisions. Optimal inlining is necessary for identifying and exploiting missed opportunities and evaluating the state of the art. This paper fills this gap through an extensive empirical analysis of optimal inlining using the SPEC2017 benchmark suite. Our novel formulation drastically reduces the inlining search space size (from 2349 down to 225) and allows us to exhaustively evaluate all inlining choices on 1,135 SPEC2017 files. We show a significant gap between the state-of-the-art strategy in LLVM and optimal inlining when optimizing for binary size, an important, deterministic metric independent of workload (in contrast to performance, another important metric). Inspired by our analysis, we introduce a simple, effective autotuning strategy for inlining that outperforms the state of the art by 7% on average (and up to 28%) on SPEC2017, 15% on the source code of LLVM itself, and 10% on the source code of SQLite. This work highlights the importance of exploring optimal inlining by providing new, actionable insight and an effective autotuning strategy that is of practical utility. @InProceedings{ASPLOS22p977, author = {Theodoros Theodoridis and Tobias Grosser and Zhendong Su}, title = {Understanding and Exploiting Optimal Function Inlining}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {977--989}, doi = {10.1145/3503222.3507744}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Guo, Linsong	ASPLOS '22: "FaaSFlow: Enable Efficient ..." FaaSFlow: Enable Efficient Workflow Execution for Function-as-a-Service Zijun Li, Yushi Liu, Linsong Guo, Quan Chen, Jiagan Cheng, Wenli Zheng, and Minyi Guo (Shanghai Jiao Tong University, China) Serverless computing (Function-as-a-Service) provides fine-grain resource sharing by running functions (or Lambdas) in containers. Data-dependent functions are required to be invoked following a pre-defined logic, which is known as serverless workflows. However, our investigation shows that the traditional master-worker based workflow execution architecture performs poorly in serverless context. One significant overhead results from the master-side workflow schedule pattern, with which the functions are triggered in the master node and assigned to worker nodes for execution. Besides, the data movement between workers also reduces the throughput. To this end, we present a worker-side workflow schedule pattern for serverless workflow execution. Following the design, we implement FaaSFlow to enable efficient workflow execution in the serverless context. Besides, we propose an adaptive storage library FaaStore that enables fast data transfer between functions on the same node without through the database. Experiment results show that FaaSFlow effectively mitigates the workflow scheduling overhead by 74.6% on average and data transmission overhead by 95% at most. When the network bandwidth fluctuates, FaaSFlow-FaaStore reduces the throughput degradation by 23.0%, and is able to multiply the utilization of network bandwidth by 1.5X-4X. @InProceedings{ASPLOS22p782, author = {Zijun Li and Yushi Liu and Linsong Guo and Quan Chen and Jiagan Cheng and Wenli Zheng and Minyi Guo}, title = {FaaSFlow: Enable Efficient Workflow Execution for Function-as-a-Service}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {782--796}, doi = {10.1145/3503222.3507717}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Guo, Minyi	ASPLOS '22: "VELTAIR: Towards High-Performance ..." VELTAIR: Towards High-Performance Multi-tenant Deep Learning Services via Adaptive Compilation and Scheduling Zihan Liu, Jingwen Leng, Zhihui Zhang, Quan Chen, Chao Li, and Minyi Guo (Shanghai Jiao Tong University, China; Shanghai Qi Zhi Institute, China) Deep learning (DL) models have achieved great success in many application domains. As such, many industrial companies such as Google and Facebook have acknowledged the importance of multi-tenant DL services. Although the multi-tenant service has been studied in conventional workloads, it is not been deeply studied on deep learning service, especially on general-purpose hardware. In this work, we systematically analyze the opportunities and challenges of providing multi-tenant deep learning services on the general-purpose CPU architecture from the aspects of scheduling granularity and code generation. We propose an adaptive granularity scheduling scheme to both guarantee resource usage efficiency and reduce the scheduling conflict rate. We also propose an adaptive compilation strategy, by which we can dynamically and intelligently pick a program with proper exclusive and shared resource usage to reduce overall interference-induced performance loss. Compared to the existing works, our design can serve more requests under the same QoS target in various scenarios (e.g., +71%, +62%, +45% for light, medium, and heavy workloads, respectively), and reduce the averaged query latency by 50%. @InProceedings{ASPLOS22p388, author = {Zihan Liu and Jingwen Leng and Zhihui Zhang and Quan Chen and Chao Li and Minyi Guo}, title = {VELTAIR: Towards High-Performance Multi-tenant Deep Learning Services via Adaptive Compilation and Scheduling}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {388--401}, doi = {10.1145/3503222.3507752}, year = {2022}, } Publisher's Version ASPLOS '22: "Astraea: Towards QoS-Aware ..." Astraea: Towards QoS-Aware and Resource-Efficient Multi-stage GPU Services Wei Zhang, Quan Chen, Kaihua Fu, Ningxin Zheng, Zhiyi Huang, Jingwen Leng, and Minyi Guo (Shanghai Jiao Tong University, China; Microsoft Research, China; University of Otago, New Zealand) Multi-stage user-facing applications on GPUs are widely-used nowa- days, and are often implemented to be microservices. Prior re- search works are not applicable to ensuring QoS of GPU-based microservices due to the different communication patterns and shared resource contentions. We propose Astraea to manage GPU microservices considering the above factors. In Astraea, a microser- vice deployment policy is used to maximize the supported peak service load while ensuring the required QoS. To adaptively switch the communication methods between microservices according to different deployments, we propose an auto-scaling GPU communi- cation framework. The framework automatically scales based on the currently used hardware topology and microservice location, and adopts global memory-based techniques to reduce intra-GPU communication. Astraea increases the supported peak load by up to 82.3% while achieving the desired 99%-ile latency target compared with state-of-the-art solutions. @InProceedings{ASPLOS22p570, author = {Wei Zhang and Quan Chen and Kaihua Fu and Ningxin Zheng and Zhiyi Huang and Jingwen Leng and Minyi Guo}, title = {Astraea: Towards QoS-Aware and Resource-Efficient Multi-stage GPU Services}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {570--582}, doi = {10.1145/3503222.3507721}, year = {2022}, } Publisher's Version ASPLOS '22: "FaaSFlow: Enable Efficient ..." FaaSFlow: Enable Efficient Workflow Execution for Function-as-a-Service Zijun Li, Yushi Liu, Linsong Guo, Quan Chen, Jiagan Cheng, Wenli Zheng, and Minyi Guo (Shanghai Jiao Tong University, China) Serverless computing (Function-as-a-Service) provides fine-grain resource sharing by running functions (or Lambdas) in containers. Data-dependent functions are required to be invoked following a pre-defined logic, which is known as serverless workflows. However, our investigation shows that the traditional master-worker based workflow execution architecture performs poorly in serverless context. One significant overhead results from the master-side workflow schedule pattern, with which the functions are triggered in the master node and assigned to worker nodes for execution. Besides, the data movement between workers also reduces the throughput. To this end, we present a worker-side workflow schedule pattern for serverless workflow execution. Following the design, we implement FaaSFlow to enable efficient workflow execution in the serverless context. Besides, we propose an adaptive storage library FaaStore that enables fast data transfer between functions on the same node without through the database. Experiment results show that FaaSFlow effectively mitigates the workflow scheduling overhead by 74.6% on average and data transmission overhead by 95% at most. When the network bandwidth fluctuates, FaaSFlow-FaaStore reduces the throughput degradation by 23.0%, and is able to multiply the utilization of network bandwidth by 1.5X-4X. @InProceedings{ASPLOS22p782, author = {Zijun Li and Yushi Liu and Linsong Guo and Quan Chen and Jiagan Cheng and Wenli Zheng and Minyi Guo}, title = {FaaSFlow: Enable Efficient Workflow Execution for Function-as-a-Service}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {782--796}, doi = {10.1145/3503222.3507717}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Guo, Zhiyuan	ASPLOS '22: "Clio: A Hardware-Software ..." Clio: A Hardware-Software Co-Designed Disaggregated Memory System Zhiyuan Guo, Yizhou Shan, Xuhao Luo, Yutong Huang, and Yiying Zhang (University of California at San Diego, USA) Memory disaggregation has attracted great attention recently because of its benefits in efficient memory utilization and ease of management. So far, memory disaggregation research has all taken one of two approaches: building/emulating memory nodes using regular servers or building them using raw memory devices with no processing power. The former incurs higher monetary cost and faces tail latency and scalability limitations, while the latter introduces performance, security, and management problems. Server-based memory nodes and memory nodes with no processing power are two extreme approaches. We seek a sweet spot in the middle by proposing a hardware-based memory disaggregation solution that has the right amount of processing power at memory nodes. Furthermore, we take a clean-slate approach by starting from the requirements of memory disaggregation and designing a memory-disaggregation-native system. We built Clio, a disaggregated memory system that virtualizes, protects, and manages disaggregated memory at hardware-based memory nodes. The Clio hardware includes a new virtual memory system, a customized network system, and a framework for computation offloading. In building Clio, we not only co-design OS functionalities, hardware architecture, and the network system, but also co-design compute nodes and memory nodes. Our FPGA prototype of Clio demonstrates that each memory node can achieve 100 Gbps throughput and an end-to-end latency of 2.5 µ s at median and 3.2 µ s at the 99th percentile. Clio also scales much better and has orders of magnitude lower tail latency than RDMA. It has 1.1× to 3.4× energy saving compared to CPU-based and SmartNIC-based disaggregated memory systems and is 2.7× faster than software-based SmartNIC solutions. @InProceedings{ASPLOS22p417, author = {Zhiyuan Guo and Yizhou Shan and Xuhao Luo and Yutong Huang and Yiying Zhang}, title = {Clio: A Hardware-Software Co-Designed Disaggregated Memory System}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {417--433}, doi = {10.1145/3503222.3507762}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional
	Haas, Sebastian	ASPLOS '22: "Efficient and Scalable Core ..." Efficient and Scalable Core Multiplexing with M³v Nils Asmussen, Sebastian Haas, Carsten Weinhold, Till Miemietz, and Michael Roitzsch (Barkhausen Institut, Germany) The M³ system (ASPLOS ’16) proposed a hardware/software co-design that simplifies integration between general-purpose cores and special-purpose accelerators, allowing users to easily utilize them in a unified manner. M³ is a tiled architecture, whose tiles (cores and accelerators) are partitioned between applications, such that each tile is dedicated to its own application. The M³x system (ATC ’19) extended M³ by trading off some isolation to enable coarse-grained multiplexing of tiles among multiple applications. With M³x, if source tile t₁ runs code of application p and sends a message m to destination tile t₂ while t₂ is currently not associated with p, then m is forwarded to the right place through a “slow path”, via some special OS tile. In this paper, we present M³v, which extends M³x by further trading off some isolation between applications to support “fast path” communication that does not require the said OS tile’s involvement. Thus, with M³v, a tile can be efficiently multiplexed between applications provided it is a general-purpose core. M³v achieves this goal by 1) adding a local multiplexer to each such core, and by 2) virtualizing the core’s hardware component responsible for cross-tile communications. We prototype M³v using RISC-V cores on an FPGA platform and show that it significantly outperforms M³x and may achieve competitive performance to Linux. @InProceedings{ASPLOS22p452, author = {Nils Asmussen and Sebastian Haas and Carsten Weinhold and Till Miemietz and Michael Roitzsch}, title = {Efficient and Scalable Core Multiplexing with M³v}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {452--466}, doi = {10.1145/3503222.3507741}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Hao, Yueming	ASPLOS '22: "ValueExpert: Exploring Value ..." ValueExpert: Exploring Value Patterns in GPU-Accelerated Applications Keren Zhou, Yueming Hao, John Mellor-Crummey, Xiaozhu Meng, and Xu Liu (Rice University, USA; North Carolina State University, USA) General-purpose GPUs have become common in modern computing systems to accelerate applications in many domains, including machine learning, high-performance computing, and autonomous driving. However, inefficiencies abound in GPU-accelerated applications, which prevent them from obtaining bare-metal performance. Performance tools play an important role in understanding performance inefficiencies in complex code bases. Many GPU performance tools pinpoint time-consuming code and provide high-level performance insights but overlook one important performance issue---value-related inefficiencies, which exist in many GPU code bases. In this paper, we present ValueExpert, a novel tool to pinpoint value-related inefficiencies in GPU applications. ValueExpert monitors application execution to capture values produced and used by each load and store operation in GPU kernels, recognizes multiple value patterns, and provides intuitive optimization guidance. We address systemic challenges in collecting, maintaining, and analyzing voluminous performance data from many GPU threads to make ValueExpert applicable to complex applications. We evaluate ValueExpert on a wide range of well-tuned benchmarks and applications, including PyTorch, Darknet, LAMMPS, Castro, and many others. ValueExpert is able to identify previously unknown performance issues and provide suggestions for nontrivial performance improvements with typically less than five lines of code changes. We verify our optimizations with application developers and upstream fixes to their repositories. @InProceedings{ASPLOS22p171, author = {Keren Zhou and Yueming Hao and John Mellor-Crummey and Xiaozhu Meng and Xu Liu}, title = {ValueExpert: Exploring Value Patterns in GPU-Accelerated Applications}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {171--185}, doi = {10.1145/3503222.3507708}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Hardavellas, Nikos	ASPLOS '22: "CARAT CAKE: Replacing Paging ..." CARAT CAKE: Replacing Paging via Compiler/Kernel Cooperation Brian Suchy, Souradip Ghosh, Drew Kersnar, Siyuan Chai, Zhen Huang, Aaron Nelson, Michael Cuevas, Alex Bernat, Gaurav Chaudhary, Nikos Hardavellas, Simone Campanoni, and Peter Dinda (Northwestern University, USA) Virtual memory, specifically paging, is undergoing significant innovation due to being challenged by new demands from modern workloads. Recent work has demonstrated an alternative software only design that can result in simplified hardware requirements, even supporting purely physical addressing. While we have made the case for this Compiler- And Runtime-based Address Translation (CARAT) concept, its evaluation was based on a user-level prototype. We now report on incorporating CARAT into a kernel, forming Compiler- And Runtime-based Address Translation for CollAborative Kernel Environments (CARAT CAKE). In our implementation, a Linux-compatible x64 process abstraction can be based either on CARAT CAKE, or on a sophisticated paging implementation. Implementing CARAT CAKE involves kernel changes and compiler optimizations/transformations that must work on all code in the system, including kernel code. We evaluate CARAT CAKE in comparison with paging and find that CARAT CAKE is able to achieve the functionality of paging (protection, mapping, and movement properties) with minimal overhead. In turn, CARAT CAKE allows significant new benefits for systems including energy savings, larger L1 caches, and arbitrary granularity memory management. @InProceedings{ASPLOS22p98, author = {Brian Suchy and Souradip Ghosh and Drew Kersnar and Siyuan Chai and Zhen Huang and Aaron Nelson and Michael Cuevas and Alex Bernat and Gaurav Chaudhary and Nikos Hardavellas and Simone Campanoni and Peter Dinda}, title = {CARAT CAKE: Replacing Paging via Compiler/Kernel Cooperation}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {98--114}, doi = {10.1145/3503222.3507771}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available
	He, Shuibing	ASPLOS '22: "NVAlloc: Rethinking Heap Metadata ..." NVAlloc: Rethinking Heap Metadata Management in Persistent Memory Allocators Zheng Dang, Shuibing He, Peiyi Hong, Zhenxin Li, Xuechen Zhang, Xian-He Sun, and Gang Chen (Zhejiang University, China; Washington State University, USA; Illinois Institute of Technology, USA) Persistent memory allocation is a fundamental building block for developing high-performance and in-memory applications. Existing persistent memory allocators suffer from suboptimal heap organizations that introduce repeated cache line flushes and small random accesses in persistent memory. Worse, many allocators use static slab segregation resulting in a dramatic increase in memory consumption when allocation request size is changed. In this paper, we design a novel allocator, named NVAlloc, to solve the above issues simultaneously. First, NVAlloc eliminates cache line reflushes by mapping contiguous data blocks in slabs to interleaved metadata entries stored in different cache lines. Second, it writes small metadata units to a persistent bookkeeping log in a sequential pattern to remove random heap metadata accesses in persistent memory. Third, instead of using static slab segregation, it supports slab morphing, which allows slabs to be transformed between size classes to significantly improve slab usage. NVAlloc is complementary to the existing consistency models. Results on 6 benchmarks demonstrate that NVAlloc improves the performance of state-of-the-art persistent memory allocators by up to 6.4x and 57x for small and large allocations, respectively. Using NVAlloc reduces memory usage by up to 57.8%. Besides, we integrate NVAlloc in a persistent FPTree. Compared to the state-of-the-art allocators, NVAlloc improves the performance of this application by up to 3.1x. @InProceedings{ASPLOS22p115, author = {Zheng Dang and Shuibing He and Peiyi Hong and Zhenxin Li and Xuechen Zhang and Xian-He Sun and Gang Chen}, title = {NVAlloc: Rethinking Heap Metadata Management in Persistent Memory Allocators}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {115--127}, doi = {10.1145/3503222.3507743}, year = {2022}, } Publisher's Version
	He, Zhenhao	ASPLOS '22: "Enzian: An Open, General, ..." Enzian: An Open, General, CPU/FPGA Platform for Systems Software Research David Cock, Abishek Ramdas, Daniel Schwyn, Michael Giardino, Adam Turowski, Zhenhao He, Nora Hossle, Dario Korolija, Melissa Licciardello, Kristina Martsenko, Reto Achermann, Gustavo Alonso, and Timothy Roscoe (ETH Zurich, Switzerland; University of British Columbia, Canada) Hybrid computing platforms, comprising CPU cores and FPGA logic, are increasingly used for accelerating data-intensive workloads in cloud deployments, and are a growing topic of interest in systems research. However, from a research perspective, existing hardware platforms are limited: they are often optimized for concrete, narrow use-cases and, therefore lack the flexibility needed to explore other applications and configurations. We show that a research group can design and build a more general, open, and affordable hardware platform for hybrid systems research. The platform, Enzian, is capable of duplicating the functionality of existing CPU/FPGA systems with comparable performance but in an open, flexible system. It couples a large FPGA with a server-class CPU in an asymmetric cache-coherent NUMA system. Enzian also enables research not possible with existing hybrid platforms, through explicit access to coherence messages, extensive thermal and power instrumentation, and an open, programmable baseboard management processor. Enzian is already being used in multiple projects, is open source (both hardware and software), and available for remote use. We present the design principles of Enzian, the challenges in building it, and evaluate it with a range of existing research use-cases alongside other, more specialized platforms, as well as demonstrating research not possible on existing platforms. @InProceedings{ASPLOS22p434, author = {David Cock and Abishek Ramdas and Daniel Schwyn and Michael Giardino and Adam Turowski and Zhenhao He and Nora Hossle and Dario Korolija and Melissa Licciardello and Kristina Martsenko and Reto Achermann and Gustavo Alonso and Timothy Roscoe}, title = {Enzian: An Open, General, CPU/FPGA Platform for Systems Software Research}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {434--451}, doi = {10.1145/3503222.3507742}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Heo, Tejun	ASPLOS '22: "IOCost: Block IO Control for ..." IOCost: Block IO Control for Containers in Datacenters Tejun Heo, Dan Schatzberg, Andrew Newell, Song Liu, Saravanan Dhakshinamurthy, Iyswarya Narayanan, Josef Bacik, Chris Mason, Chunqiang Tang, and Dimitrios Skarlatos (Meta, USA; Carnegie Mellon University, USA) Resource isolation is a fundamental requirement in datacenter environments. However, our production experience in Meta’s large-scale datacenters shows that existing IO control mechanisms for block storage are inadequate in containerized environments. IO control needs to provide proportional resources to containers while taking into account the hardware heterogeneity of storage devices and the idiosyncrasies of the workloads deployed in datacenters. The speed of modern SSDs requires IO control to execute with low-overheads. Furthermore, IO control should strive for work conservation, take into account the interactions with the memory management subsystem, and avoid priority inversions that lead to isolation failures. To address these challenges, this paper presents IOCost, an IO control solution that is designed for containerized environments and provides scalable, work-conserving, and low-overhead IO control for heterogeneous storage devices and diverse workloads in datacenters. IOCost performs offline profiling to build a device model and uses it to estimate device occupancy of each IO request. To minimize runtime overhead, it separates IO control into a fast per-IO issue path and a slower periodic planning path. A novel work-conserving budget donation algorithm enables containers to dynamically share unused budget. We have deployed IOCost across the entirety of Meta’s datacenters comprised of millions of ma- chines, upstreamed IOCost to the Linux kernel, and open-sourced our device-profiling tools. IOCost has been running in production for two years, providing IO control for Meta’s fleet. We describe the design of IOCost and share our experience deploying it at scale. @InProceedings{ASPLOS22p595, author = {Tejun Heo and Dan Schatzberg and Andrew Newell and Song Liu and Saravanan Dhakshinamurthy and Iyswarya Narayanan and Josef Bacik and Chris Mason and Chunqiang Tang and Dimitrios Skarlatos}, title = {IOCost: Block IO Control for Containers in Datacenters}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {595--608}, doi = {10.1145/3503222.3507727}, year = {2022}, } Publisher's Version ASPLOS '22: "TMO: Transparent Memory Offloading ..." TMO: Transparent Memory Offloading in Datacenters Johannes Weiner, Niket Agarwal, Dan Schatzberg, Leon Yang, Hao Wang, Blaise Sanouillet, Bikash Sharma, Tejun Heo, Mayank Jain, Chunqiang Tang, and Dimitrios Skarlatos (Meta, USA; Carnegie Mellon University, USA) The unrelenting growth of the memory needs of emerging datacenter applications, along with ever increasing cost and volatility of DRAM prices, has led to DRAM being a major infrastructure expense. Alternative technologies, such as NVMe SSDs and upcoming NVM devices, offer higher capacity than DRAM at a fraction of the cost and power. One promising approach is to transparently offload colder memory to cheaper memory technologies via kernel or hypervisor techniques. The key challenge, however, is to develop a datacenter-scale solution that is robust in dealing with diverse workloads and large performance variance of different offload devices such as compressed memory, SSD, and NVM. This paper presents TMO, Meta’s transparent memory offloading solution for heterogeneous datacenter environments. TMO introduces a new Linux kernel mechanism that directly measures in realtime the lost work due to resource shortage across CPU, memory, and I/O. Guided by this information and without any prior application knowledge, TMO automatically adjusts how much memory to offload to heterogeneous devices (e.g., compressed memory or SSD) according to the device’s performance characteristics and the application’s sensitivity to memory-access slowdown. TMO holistically identifies offloading opportunities from not only the application containers but also the sidecar containers that provide infrastructure-level functions. To maximize memory savings, TMO targets both anonymous memory and file cache, and balances the swap-in rate of anonymous memory and the reload rate of file pages that were recently evicted from the file cache. TMO has been running in production for more than a year, and has saved between 20-32% of the total memory across millions of servers in our large datacenter fleet. We have successfully upstreamed TMO into the Linux kernel. @InProceedings{ASPLOS22p609, author = {Johannes Weiner and Niket Agarwal and Dan Schatzberg and Leon Yang and Hao Wang and Blaise Sanouillet and Bikash Sharma and Tejun Heo and Mayank Jain and Chunqiang Tang and Dimitrios Skarlatos}, title = {TMO: Transparent Memory Offloading in Datacenters}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {609--621}, doi = {10.1145/3503222.3507731}, year = {2022}, } Publisher's Version
	Hermony, Amit	ASPLOS '22: "FlexDriver: A Network Driver ..." FlexDriver: A Network Driver for Your Accelerator Haggai Eran, Maxim Fudim, Gabi Malka, Gal Shalom, Noam Cohen, Amit Hermony, Dotan Levi, Liran Liss, and Mark Silberstein (NVIDIA, Israel; Technion, Israel) We propose a new system design for connecting hardware and FPGA accelerators to the network, allowing the accelerator to directly control commodity Network Interface Cards (NICs) without using the CPU. This enables us to solve the key challenge of leveraging existing NIC hardware offloads such as virtualization, tunneling, and RDMA for accelerator networking. Our approach supports a diverse set of use cases, from direct network access for disaggregated accelerators to inline-acceleration of the network stack, all without the complex networking logic in the accelerator. To demonstrate the feasibility of this approach, we build FlexDriver (FLD), an on-accelerator hardware module that implements a NIC data-plane driver. Our main technical contribution is a mechanism that compresses the NIC control structures by two orders of magnitude, allowing FLD to achieve high networking scalability with low die area cost and no bandwidth interference with the accelerator logic. The prototype for NVIDIA Innova-2 FPGA SmartNICs showcases our design’s utility for three different accelerators: a disaggregated LTE cipher, an IP-defragmentation inline accelerator, and an IoT cryptographic-token authentication offload. These accelerators reach 25 Gbps line rate and leverage the NIC for RDMA processing, VXLAN tunneling, and traffic shaping without CPU involvement. @InProceedings{ASPLOS22p1115, author = {Haggai Eran and Maxim Fudim and Gabi Malka and Gal Shalom and Noam Cohen and Amit Hermony and Dotan Levi and Liran Liss and Mark Silberstein}, title = {FlexDriver: A Network Driver for Your Accelerator}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1115--1129}, doi = {10.1145/3503222.3507776}, year = {2022}, } Publisher's Version
	Hicks, Matthew	ASPLOS '22: "SRAM Has No Chill: Exploiting ..." SRAM Has No Chill: Exploiting Power Domain Separation to Steal On-Chip Secrets Jubayer Mahmod and Matthew Hicks (Virginia Tech, USA) The abundance of embedded systems and smart devices increases the risk of physical memory disclosure attacks. One such classic non-invasive attack exploits dynamic RAM's temperature-dependent ability to retain information across power cycles---known as a cold boot attack. When exposed to low temperatures, DRAM cells preserve their state for a short time without power, mimicking non-volatile memories in that time frame. Attackers exploit this physical phenomenon to gain access to a system's secrets, leading to data theft from encrypted storage. To prevent cold boot attacks, programmers hide secrets on-chip in Static Random-Access Memory (SRAM); by construction, on-chip SRAM is isolated from external probing and has little intrinsic capacitance, making it robust against cold boot attacks. While it is the case that SRAM protects against traditional cold boot attacks, we show that there is another way to retain information in on-chip SRAM across power cycles and software changes. This paper presents Volt Boot, an attack that demonstrates a vulnerability of on-chip volatile memories due to the physical separation common to modern system-on-chip power distribution networks. Volt Boot leverages asymmetrical power states (e.g., on vs. off) to force SRAM state retention across power cycles, eliminating the need for traditional cold boot attack enablers, such as low-temperature or intrinsic data retention time. Using several modern ARM Cortex-A devices, we demonstrate the effectiveness of the attack in caches, registers, and iRAMs. Unlike other forms of SRAM data retention attacks, Volt Boot retrieves data with 100% accuracy---without any complex post-processing. @InProceedings{ASPLOS22p1043, author = {Jubayer Mahmod and Matthew Hicks}, title = {SRAM Has No Chill: Exploiting Power Domain Separation to Steal On-Chip Secrets}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1043--1055}, doi = {10.1145/3503222.3507710}, year = {2022}, } Publisher's Version ASPLOS '22: "Invisible Bits: Hiding Secret ..." Invisible Bits: Hiding Secret Messages in SRAM’s Analog Domain Jubayer Mahmod and Matthew Hicks (Virginia Tech, USA) Electronic devices are increasingly the subject of inspection by authorities. While encryption hides secret messages, it does not hide the transmission of those secret messages---in fact, it calls attention to them. Thus, an adversary, seeing encrypted data, turns to coercion to extract the credentials required to reveal the secret message. Steganographic techniques hide secret messages in plain sight, providing the user with plausible deniability, removing the threat of coercion. This paper unveils Invisible Bits a new steganographic technique that hides secret messages in the analog domain of Static Random Access Memory (SRAM) embedded within a computing device. Unlike other memory technologies, the power-on state of SRAM reveals the analog-domain properties of its individual cells. We show how to quickly and systematically change the analog-domain properties of SRAM cells to encode data in the analog domain and how to reveal those changes by capturing SRAM's power-on state. Experiments with commercial devices show that Invisible Bits provides over 90% capacity---two orders-of-magnitude more than previous on-chip steganographic approaches, while retaining device functionality---even when the device undergoes subsequent normal operation or is shelved for months. Experiments also show that adversaries cannot differentiate between devices with encoded messages and those without. Lastly, we show how to layer encryption and error correction on top of our message encoding scheme in an end-to-end demonstration. @InProceedings{ASPLOS22p1086, author = {Jubayer Mahmod and Matthew Hicks}, title = {Invisible Bits: Hiding Secret Messages in SRAM’s Analog Domain}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1086--1098}, doi = {10.1145/3503222.3507756}, year = {2022}, } Publisher's Version
	Hoffmann, Henry	ASPLOS '22: "Protecting Adaptive Sampling ..." Protecting Adaptive Sampling from Information Leakage on Low-Power Sensors Tejas Kannan and Henry Hoffmann (University of Chicago, USA) Adaptive sampling is a powerful family of algorithms for managing energy consumption on low-power sensors. These algorithms use captured measurements to control the sensor's collection rate, leading to near-optimal error under energy constraints. Adaptive sampling's data-driven nature, however, comes at a cost in privacy. In this work, we demonstrate how the collection rates of general adaptive policies leak information about captured measurements. Further, individual adaptive policies display this leakage on multiple tasks. This result presents a challenge in maintaining privacy for sensors using energy-efficient batched communication. In this context, the size of measurement batches exposes the sampling policy's collection rate. Thus, an attacker who monitors the encrypted link between sensor and server can use message lengths to uncover information about the captured values. We address this side-channel by introducing a framework called Adaptive Group Encoding (AGE) that protects any periodic adaptive sampler. AGE uses quantization to encode all batches as fixed-length messages, making message sizes independent of the collection rate. AGE reduces the quantization error through a series of transformations. The proposed framework preserves the low error of adaptive sampling while preventing information leakage and incurring negligible energy overhead. @InProceedings{ASPLOS22p240, author = {Tejas Kannan and Henry Hoffmann}, title = {Protecting Adaptive Sampling from Information Leakage on Low-Power Sensors}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {240--254}, doi = {10.1145/3503222.3507775}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Hofmann, Matthew	ASPLOS '22: "PLD: Fast FPGA Compilation ..." PLD: Fast FPGA Compilation to Make Reconfigurable Acceleration Compatible with Modern Incremental Refinement Software Development Yuanlong Xiao, Eric Micallef, Andrew Butt, Matthew Hofmann, Marc Alston, Matthew Goldsmith, Andrew Merczynski-Hait, and André DeHon (University of Pennsylvania, USA) FPGA-based accelerators are demonstrating significant absolute performance and energy efficiency compared with general-purpose CPUs. While FPGA computations can now be described in standard, programming languages, like C, development for FPGAs accelerators remains tedious and inaccessible to modern software engineers. Slow compiles (potentially taking tens of hours) inhibit the rapid, incremental refinement of designs that is the hallmark of modern software engineering. To address this issue, we introduce separate compilation and linkage into the FPGA design flow, providing faster design turns more familiar to software development. To realize this flow, we provide abstractions, compiler options, and compiler flow that allow the same C source code to be compiled to processor cores in seconds and to FPGA regions in minutes, providing the missing -O0 and -O1 options familiar in software development. This raises the FPGA programming level and standardizes the programming experience, bringing FPGA-based accelerators into a more familiar software platform ecosystem for software engineers. @InProceedings{ASPLOS22p933, author = {Yuanlong Xiao and Eric Micallef and Andrew Butt and Matthew Hofmann and Marc Alston and Matthew Goldsmith and Andrew Merczynski-Hait and André DeHon}, title = {PLD: Fast FPGA Compilation to Make Reconfigurable Acceleration Compatible with Modern Incremental Refinement Software Development}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {933--945}, doi = {10.1145/3503222.3507740}, year = {2022}, } Publisher's Version
	Hong, Peiyi	ASPLOS '22: "NVAlloc: Rethinking Heap Metadata ..." NVAlloc: Rethinking Heap Metadata Management in Persistent Memory Allocators Zheng Dang, Shuibing He, Peiyi Hong, Zhenxin Li, Xuechen Zhang, Xian-He Sun, and Gang Chen (Zhejiang University, China; Washington State University, USA; Illinois Institute of Technology, USA) Persistent memory allocation is a fundamental building block for developing high-performance and in-memory applications. Existing persistent memory allocators suffer from suboptimal heap organizations that introduce repeated cache line flushes and small random accesses in persistent memory. Worse, many allocators use static slab segregation resulting in a dramatic increase in memory consumption when allocation request size is changed. In this paper, we design a novel allocator, named NVAlloc, to solve the above issues simultaneously. First, NVAlloc eliminates cache line reflushes by mapping contiguous data blocks in slabs to interleaved metadata entries stored in different cache lines. Second, it writes small metadata units to a persistent bookkeeping log in a sequential pattern to remove random heap metadata accesses in persistent memory. Third, instead of using static slab segregation, it supports slab morphing, which allows slabs to be transformed between size classes to significantly improve slab usage. NVAlloc is complementary to the existing consistency models. Results on 6 benchmarks demonstrate that NVAlloc improves the performance of state-of-the-art persistent memory allocators by up to 6.4x and 57x for small and large allocations, respectively. Using NVAlloc reduces memory usage by up to 57.8%. Besides, we integrate NVAlloc in a persistent FPTree. Compared to the state-of-the-art allocators, NVAlloc improves the performance of this application by up to 3.1x. @InProceedings{ASPLOS22p115, author = {Zheng Dang and Shuibing He and Peiyi Hong and Zhenxin Li and Xuechen Zhang and Xian-He Sun and Gang Chen}, title = {NVAlloc: Rethinking Heap Metadata Management in Persistent Memory Allocators}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {115--127}, doi = {10.1145/3503222.3507743}, year = {2022}, } Publisher's Version
	Hossle, Nora	ASPLOS '22: "Enzian: An Open, General, ..." Enzian: An Open, General, CPU/FPGA Platform for Systems Software Research David Cock, Abishek Ramdas, Daniel Schwyn, Michael Giardino, Adam Turowski, Zhenhao He, Nora Hossle, Dario Korolija, Melissa Licciardello, Kristina Martsenko, Reto Achermann, Gustavo Alonso, and Timothy Roscoe (ETH Zurich, Switzerland; University of British Columbia, Canada) Hybrid computing platforms, comprising CPU cores and FPGA logic, are increasingly used for accelerating data-intensive workloads in cloud deployments, and are a growing topic of interest in systems research. However, from a research perspective, existing hardware platforms are limited: they are often optimized for concrete, narrow use-cases and, therefore lack the flexibility needed to explore other applications and configurations. We show that a research group can design and build a more general, open, and affordable hardware platform for hybrid systems research. The platform, Enzian, is capable of duplicating the functionality of existing CPU/FPGA systems with comparable performance but in an open, flexible system. It couples a large FPGA with a server-class CPU in an asymmetric cache-coherent NUMA system. Enzian also enables research not possible with existing hybrid platforms, through explicit access to coherence messages, extensive thermal and power instrumentation, and an open, programmable baseboard management processor. Enzian is already being used in multiple projects, is open source (both hardware and software), and available for remote use. We present the design principles of Enzian, the challenges in building it, and evaluate it with a range of existing research use-cases alongside other, more specialized platforms, as well as demonstrating research not possible on existing platforms. @InProceedings{ASPLOS22p434, author = {David Cock and Abishek Ramdas and Daniel Schwyn and Michael Giardino and Adam Turowski and Zhenhao He and Nora Hossle and Dario Korolija and Melissa Licciardello and Kristina Martsenko and Reto Achermann and Gustavo Alonso and Timothy Roscoe}, title = {Enzian: An Open, General, CPU/FPGA Platform for Systems Software Research}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {434--451}, doi = {10.1145/3503222.3507742}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Hu, Hong	ASPLOS '22: "Who Goes First? Detecting ..." Who Goes First? Detecting Go Concurrency Bugs via Message Reordering Ziheng Liu, Shihao Xia, Yu Liang, Linhai Song, and Hong Hu (Pennsylvania State University, USA) Go is a young programming language invented to build safe and efficient concurrent programs. It provides goroutines as lightweight threads and channels for inter-goroutine communication. Programmers are encouraged to explicitly pass messages through channels to connect goroutines, with the purpose of reducing the chance of making programming mistakes and introducing concurrency bugs. Go is one of the most beloved programming languages and has already been used to build many critical infrastructure software systems in the data-center environment. However, a recent study shows that channel-related concurrency bugs are still common in Go programs, severely hurting the reliability of the programs. This paper presents GFuzz, a dynamic detector that can effectively pinpoint channel-related concurrency bugs by mutating the processing orders of concurrent messages. We build GFuzz in three steps. We first adopt an effective approach to identify concurrent messages and transform a program to process those messages in any given order. We then take a fuzzing approach to generate new processing orders by mutating exercised ones and rely on execution feedback to prioritize orders close to triggering bugs. Finally, we design a runtime sanitizer to capture triggered bugs that are missed by the Go runtime. We evaluate GFuzz on seven popular Go software systems, including Docker, Kubernetes, and gRPC. GFuzz finds 184 previously unknown bugs and reports a negligible number of false positives. Programmers have already confirmed 124 reports as real bugs and fixed 67 of them based on our reporting. A careful inspection of the detected concurrency bugs from gRPC shows the effectiveness of each component of GFuzz and confirms the components' rationality. @InProceedings{ASPLOS22p888, author = {Ziheng Liu and Shihao Xia and Yu Liang and Linhai Song and Hong Hu}, title = {Who Goes First? Detecting Go Concurrency Bugs via Message Reordering}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {888--902}, doi = {10.1145/3503222.3507753}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional
	Hu, Shi-Min	ASPLOS '22: "Path-Sensitive and Alias-Aware ..." Path-Sensitive and Alias-Aware Typestate Analysis for Detecting OS Bugs Tuo Li, Jia-Ju Bai, Yulei Sui, and Shi-Min Hu (Tsinghua University, China; University of Technology Sydney, Australia) Operating system (OS) is the cornerstone for modern computer systems. It manages devices and provides fundamental service for user-level applications. Thus, detecting bugs in OSes is important to improve reliability and security of computer systems. Static typestate analysis is a common technique for detecting different types of bugs, but it is often inaccurate or unscalable for large-size OS code, due to imprecision of identifying alias relationships as well as high costs of typestate tracking and path-feasibility validation. In this paper, we present PATA, a novel path-sensitive and aliasaware typestate analysis framework to detect OS bugs. To improve the precision of identifying alias relationships in OS code, PATA performs a path-based alias analysis based on control-flow paths and access paths. With these alias relationships, PATA reduces the costs of typestate tracking and path-feasibility validation, to boost the efficiency of path-sensitive typestate analysis for bug detection. We have evaluated PATA on the Linux kernel and three popular IoT OSes (Zephyr, RIOT and TencentOS-tiny) to detect three common types of bugs (null-pointer dereferences, uninitialized variable accesses and memory leaks). PATA finds 574 real bugs with a false positive rate of 28%. 206 of these bugs have been confirmed by the developers of the four OSes.We also compare PATA to seven state-of-the-art static approaches (Cppcheck, Coccinelle, Smatch,CSA, Infer, Saber and SVF). PATA finds many real bugs missed by them, with a lower false positive rate. @InProceedings{ASPLOS22p859, author = {Tuo Li and Jia-Ju Bai and Yulei Sui and Shi-Min Hu}, title = {Path-Sensitive and Alias-Aware Typestate Analysis for Detecting OS Bugs}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {859--872}, doi = {10.1145/3503222.3507770}, year = {2022}, } Publisher's Version
	Hu, Yufeng	ASPLOS '22: "EXAMINER: Automatically Locating ..." EXAMINER: Automatically Locating Inconsistent Instructions between Real Devices and CPU Emulators for ARM Muhui Jiang, Tianyi Xu, Yajin Zhou, Yufeng Hu, Ming Zhong, Lei Wu, Xiapu Luo, and Kui Ren (Hong Kong Polytechnic University, China; Zhejiang University, China) Emulators are widely used to build dynamic analysis frameworks due to its fine-grained tracing capability, full system monitoring functionality, and scalability of running on different operating systems and architectures. However, whether emulators are consistent with real devices is unknown. To understand this problem, we aim to automatically locate inconsistent instructions, which behave differently between emulators and real devices. We target the ARM architecture, which provides machine-readable specifications. Based on the specification, we propose a sufficient test case generator by designing and implementing the first symbolic execution engine for the ARM architecture specification language (ASL). We generate 2,774,649 representative instruction streams and conduct differential testing between four ARM real devices in different architecture versions (i.e., ARMv5, ARMv6, ARMv7, and ARMv8) and three state-of-the-art emulators (i.e., QEMU, Unicorn, and Angr). We locate a huge number of inconsistent instruction streams (171,858 for QEMU, 223,264 for unicorn, and 120,169 for Angr). We find that undefined implementation in ARM manual and bugs of emulators are the major causes of inconsistencies. Furthermore, we discover 12 bugs, which influence commonly used instructions (e.g., BLX). With the inconsistent instructions, we build three security applications and demonstrate the capability of these instructions on detecting emulators, anti-emulation, and anti-fuzzing. @InProceedings{ASPLOS22p846, author = {Muhui Jiang and Tianyi Xu and Yajin Zhou and Yufeng Hu and Ming Zhong and Lei Wu and Xiapu Luo and Kui Ren}, title = {EXAMINER: Automatically Locating Inconsistent Instructions between Real Devices and CPU Emulators for ARM}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {846--858}, doi = {10.1145/3503222.3507736}, year = {2022}, } Publisher's Version
	Hua, Yu	ASPLOS '22: "Efficiently Detecting Concurrency ..." Efficiently Detecting Concurrency Bugs in Persistent Memory Programs Zhangyu Chen, Yu Hua, Yongle Zhang, and Luochangqi Ding (Huazhong University of Science and Technology, China; Purdue University, USA) Due to the salient DRAM-comparable performance, TB-scale capacity, and non-volatility, persistent memory (PM) provides new opportunities for large-scale in-memory computing with instant crash recovery. However, programming PM systems is error-prone due to the existence of crash-consistency bugs, which are challenging to diagnose especially with concurrent programming widely adopted in PM applications to exploit hardware parallelism. Existing bug detection tools for DRAM-based concurrency issues cannot detect PM crash-consistency bugs because they are oblivious to PM operations and PM consistency. On the other hand, existing PM-specific debugging tools only focus on sequential PM programs and cannot effectively detect crash-consistency issues hidden in concurrent executions. In order to effectively detect crash-consistency bugs that only manifest in concurrent executions, we propose PMRace, the first PM-specific concurrency bug detection tool. We identify and define two new types of concurrent crash-consistency bugs: PM Inter-thread Inconsistency and PM Synchronization Inconsistency. In particular, PMRace adopts PM-aware and coverage-guided fuzz testing to explore PM program executions. For PM Inter-thread Inconsistency, which denotes the data inconsistency hidden in thread interleavings, PMRace performs PM-aware interleaving exploration and thread scheduling to drive the execution towards executions that reveal such inconsistencies. For PM Synchronization Inconsistency between persisted synchronization variables and program data, PMRace identifies the inconsistency during interleaving exploration. The post-failure validation reduces the false positives that come from custom crash recovery mechanisms. PMRace has found 14 bugs (10 new bugs) in real-world concurrent PM systems including PM-version memcached. @InProceedings{ASPLOS22p873, author = {Zhangyu Chen and Yu Hua and Yongle Zhang and Luochangqi Ding}, title = {Efficiently Detecting Concurrency Bugs in Persistent Memory Programs}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {873--887}, doi = {10.1145/3503222.3507755}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Huang, Jian	ASPLOS '22: "RSSD: Defend against Ransomware ..." RSSD: Defend against Ransomware with Hardware-Isolated Network-Storage Codesign and Post-Attack Analysis Benjamin Reidys, Peng Liu, and Jian Huang (University of Illinois at Urbana-Champaign, USA; Pennsylvania State University, USA) Encryption ransomware has become a notorious malware. It encrypts user data on storage devices like solid-state drives (SSDs) and demands a ransom to restore data for users. To bypass existing defenses, ransomware would keep evolving and performing new attack models. For instance, we identify and validate three new attacks, including (1) garbage-collection (GC) attack that exploits storage capacity and keeps writing data to trigger GC and force SSDs to release the retained data; (2) timing attack that intentionally slows down the pace of encrypting data and hides its I/O patterns to escape existing defense; (3) trimming attack that utilizes the trim command available in SSDs to physically erase data. To enhance the robustness of SSDs against these attacks, we propose RSSD, a ransomware-aware SSD. It redesigns the flash management of SSDs for enabling the hardware-assisted logging, which can conservatively retain older versions of user data and received storage operations in time order with low overhead. It also employs hardware-isolated NVMe over Ethernet to expand local storage capacity by transparently offloading the logs to remote cloud/servers in a secure manner. RSSD enables post-attack analysis by building a trusted evidence chain of storage operations to assist the investigation of ransomware attacks. We develop RSSD with a real-world SSD FPGA board. Our evaluation shows that RSSD can defend against new and future ransomware attacks, while introducing negligible performance overhead. @InProceedings{ASPLOS22p726, author = {Benjamin Reidys and Peng Liu and Jian Huang}, title = {RSSD: Defend against Ransomware with Hardware-Isolated Network-Storage Codesign and Post-Attack Analysis}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {726--739}, doi = {10.1145/3503222.3507773}, year = {2022}, } Publisher's Version
	Huang, Jun	ASPLOS '22: "Breaking the Computation and ..." Breaking the Computation and Communication Abstraction Barrier in Distributed Machine Learning Workloads Abhinav Jangda, Jun Huang, Guodong Liu, Amir Hossein Nodehi Sabet, Saeed Maleki, Youshan Miao, Madanlal Musuvathi, Todd Mytkowicz, and Olli Saarikivi (University of Massachusetts at Amherst, USA; Ohio State University, USA; Chinese Academy of Sciences, China; University of California at Riverside, USA; Microsoft Research, USA; Microsoft Research, China) Recent trends towards large machine learning models require both training and inference tasks to be distributed. Considering the huge cost of training these models, it is imperative to unlock optimizations in computation and communication to obtain best performance. However, the current logical separation between computation and communication kernels in machine learning frameworks misses optimization opportunities across this barrier. Breaking this abstraction can provide many optimizations to improve the performance of distributed workloads. However, manually applying these optimizations requires modifying the underlying computation and communication libraries for each scenario, which is both time consuming and error-prone. Therefore, we present CoCoNet, which contains (i) a domain specific language to express a distributed machine learning program in the form of computation and communication operations, (ii) a set of semantics preserving transformations to optimize the program, and (iii) a compiler to generate jointly optimized communication and computation GPU kernels. Providing both computation and communication as first class constructs allows users to work on a high-level abstraction and apply powerful optimizations, such as fusion or overlapping of communication and computation. CoCoNet enabled us to optimize data-, model- and pipeline-parallel workloads in large language models with only a few lines of code. Our experiments show that CoCoNet significantly outperforms state-of-the-art distributed machine learning implementations. @InProceedings{ASPLOS22p402, author = {Abhinav Jangda and Jun Huang and Guodong Liu and Amir Hossein Nodehi Sabet and Saeed Maleki and Youshan Miao and Madanlal Musuvathi and Todd Mytkowicz and Olli Saarikivi}, title = {Breaking the Computation and Communication Abstraction Barrier in Distributed Machine Learning Workloads}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {402--416}, doi = {10.1145/3503222.3507778}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional
	Huang, Kai	ASPLOS '22: "Eavesdropping User Credentials ..." Eavesdropping User Credentials via GPU Side Channels on Smartphones Boyuan Yang, Ruirong Chen, Kai Huang, Jun Yang, and Wei Gao (University of Pittsburgh, USA) Graphics Processing Unit (GPU) on smartphones is an effective target for hardware attacks. In this paper, we present a new side channel attack on mobile GPUs of Android smartphones, allowing an unprivileged attacker to eavesdrop the user's credentials, such as login usernames and passwords, from their inputs through on-screen keyboard. Our attack targets on Qualcomm Adreno GPUs and investigate the amount of GPU overdraw when rendering the popups of user's key presses of inputs. Such GPU overdraw caused by each key press corresponds to unique variations of selected GPU performance counters, from which these key presses can be accurately inferred. Experiment results from practical use on multiple models of Android smartphones show that our attack can correctly infer more than 80% of user's credential inputs, but incur negligible amounts of computing overhead and network traffic on the victim device. To counter this attack, this paper suggests mitigations of access control on GPU performance counters, or applying obfuscations on the values of GPU performance counters. @InProceedings{ASPLOS22p285, author = {Boyuan Yang and Ruirong Chen and Kai Huang and Jun Yang and Wei Gao}, title = {Eavesdropping User Credentials via GPU Side Channels on Smartphones}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {285--299}, doi = {10.1145/3503222.3507757}, year = {2022}, } Publisher's Version Published Artifact Video Artifacts Available Artifacts Functional
	Huang, Yu	ASPLOS '22: "CirFix: Automatically Repairing ..." CirFix: Automatically Repairing Defects in Hardware Design Code Hammad Ahmad, Yu Huang, and Westley Weimer (University of Michigan, USA) This paper presents CirFix, a framework for automatically repairing defects in hardware designs implemented in languages like Verilog. We propose a novel fault localization approach based on assignments to wires and registers, and a fitness function tailored to the hardware domain to bridge the gap between software-level automated program repair and hardware descriptions. We also present a benchmark suite of 32 defect scenarios corresponding to a variety of hardware projects. Overall, CirFix produces plausible repairs for 21/32 and correct repairs for 16/32 of the defect scenarios. This repair rate is comparable to that of successful program repair approaches for software, indicating CirFix is effective at bringing over the benefits of automated program repair to the hardware domain for the first time. @InProceedings{ASPLOS22p990, author = {Hammad Ahmad and Yu Huang and Westley Weimer}, title = {CirFix: Automatically Repairing Defects in Hardware Design Code}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {990--1003}, doi = {10.1145/3503222.3507763}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Huang, Yutong	ASPLOS '22: "Clio: A Hardware-Software ..." Clio: A Hardware-Software Co-Designed Disaggregated Memory System Zhiyuan Guo, Yizhou Shan, Xuhao Luo, Yutong Huang, and Yiying Zhang (University of California at San Diego, USA) Memory disaggregation has attracted great attention recently because of its benefits in efficient memory utilization and ease of management. So far, memory disaggregation research has all taken one of two approaches: building/emulating memory nodes using regular servers or building them using raw memory devices with no processing power. The former incurs higher monetary cost and faces tail latency and scalability limitations, while the latter introduces performance, security, and management problems. Server-based memory nodes and memory nodes with no processing power are two extreme approaches. We seek a sweet spot in the middle by proposing a hardware-based memory disaggregation solution that has the right amount of processing power at memory nodes. Furthermore, we take a clean-slate approach by starting from the requirements of memory disaggregation and designing a memory-disaggregation-native system. We built Clio, a disaggregated memory system that virtualizes, protects, and manages disaggregated memory at hardware-based memory nodes. The Clio hardware includes a new virtual memory system, a customized network system, and a framework for computation offloading. In building Clio, we not only co-design OS functionalities, hardware architecture, and the network system, but also co-design compute nodes and memory nodes. Our FPGA prototype of Clio demonstrates that each memory node can achieve 100 Gbps throughput and an end-to-end latency of 2.5 µ s at median and 3.2 µ s at the 99th percentile. Clio also scales much better and has orders of magnitude lower tail latency than RDMA. It has 1.1× to 3.4× energy saving compared to CPU-based and SmartNIC-based disaggregated memory systems and is 2.7× faster than software-based SmartNIC solutions. @InProceedings{ASPLOS22p417, author = {Zhiyuan Guo and Yizhou Shan and Xuhao Luo and Yutong Huang and Yiying Zhang}, title = {Clio: A Hardware-Software Co-Designed Disaggregated Memory System}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {417--433}, doi = {10.1145/3503222.3507762}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional
	Huang, Zhen	ASPLOS '22: "CARAT CAKE: Replacing Paging ..." CARAT CAKE: Replacing Paging via Compiler/Kernel Cooperation Brian Suchy, Souradip Ghosh, Drew Kersnar, Siyuan Chai, Zhen Huang, Aaron Nelson, Michael Cuevas, Alex Bernat, Gaurav Chaudhary, Nikos Hardavellas, Simone Campanoni, and Peter Dinda (Northwestern University, USA) Virtual memory, specifically paging, is undergoing significant innovation due to being challenged by new demands from modern workloads. Recent work has demonstrated an alternative software only design that can result in simplified hardware requirements, even supporting purely physical addressing. While we have made the case for this Compiler- And Runtime-based Address Translation (CARAT) concept, its evaluation was based on a user-level prototype. We now report on incorporating CARAT into a kernel, forming Compiler- And Runtime-based Address Translation for CollAborative Kernel Environments (CARAT CAKE). In our implementation, a Linux-compatible x64 process abstraction can be based either on CARAT CAKE, or on a sophisticated paging implementation. Implementing CARAT CAKE involves kernel changes and compiler optimizations/transformations that must work on all code in the system, including kernel code. We evaluate CARAT CAKE in comparison with paging and find that CARAT CAKE is able to achieve the functionality of paging (protection, mapping, and movement properties) with minimal overhead. In turn, CARAT CAKE allows significant new benefits for systems including energy savings, larger L1 caches, and arbitrary granularity memory management. @InProceedings{ASPLOS22p98, author = {Brian Suchy and Souradip Ghosh and Drew Kersnar and Siyuan Chai and Zhen Huang and Aaron Nelson and Michael Cuevas and Alex Bernat and Gaurav Chaudhary and Nikos Hardavellas and Simone Campanoni and Peter Dinda}, title = {CARAT CAKE: Replacing Paging via Compiler/Kernel Cooperation}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {98--114}, doi = {10.1145/3503222.3507771}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available
	Huang, Zhiyi	ASPLOS '22: "Astraea: Towards QoS-Aware ..." Astraea: Towards QoS-Aware and Resource-Efficient Multi-stage GPU Services Wei Zhang, Quan Chen, Kaihua Fu, Ningxin Zheng, Zhiyi Huang, Jingwen Leng, and Minyi Guo (Shanghai Jiao Tong University, China; Microsoft Research, China; University of Otago, New Zealand) Multi-stage user-facing applications on GPUs are widely-used nowa- days, and are often implemented to be microservices. Prior re- search works are not applicable to ensuring QoS of GPU-based microservices due to the different communication patterns and shared resource contentions. We propose Astraea to manage GPU microservices considering the above factors. In Astraea, a microser- vice deployment policy is used to maximize the supported peak service load while ensuring the required QoS. To adaptively switch the communication methods between microservices according to different deployments, we propose an auto-scaling GPU communi- cation framework. The framework automatically scales based on the currently used hardware topology and microservice location, and adopts global memory-based techniques to reduce intra-GPU communication. Astraea increases the supported peak load by up to 82.3% while achieving the desired 99%-ile latency target compared with state-of-the-art solutions. @InProceedings{ASPLOS22p570, author = {Wei Zhang and Quan Chen and Kaihua Fu and Ningxin Zheng and Zhiyi Huang and Jingwen Leng and Minyi Guo}, title = {Astraea: Towards QoS-Aware and Resource-Efficient Multi-stage GPU Services}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {570--582}, doi = {10.1145/3503222.3507721}, year = {2022}, } Publisher's Version
	Huda, Safeen	ASPLOS '22: "A Full-Stack Search Technique ..." A Full-Stack Search Technique for Domain Optimized Deep Learning Accelerators Dan Zhang, Safeen Huda, Ebrahim Songhori, Kartik Prabhu, Quoc Le, Anna Goldie, and Azalia Mirhoseini (Google Brain, USA; Google, USA; Stanford University, USA) The rapidly-changing deep learning landscape presents a unique opportunity for building inference accelerators optimized for specific datacenter-scale workloads. We propose Full-stack Accelerator Search Technique (FAST), a hardware accelerator search framework that defines a broad optimization environment covering key design decisions within the hardware-software stack, including hardware datapath, software scheduling, and compiler passes such as operation fusion and tensor padding. In this paper, we analyze bottlenecks in state-of-the-art vision and natural language processing (NLP) models, including EfficientNet and BERT, and use FAST to design accelerators capable of addressing these bottlenecks. FAST-generated accelerators optimized for single workloads improve Perf/TDP by 3.7× on average across all benchmarks compared to TPU-v3. A FAST-generated accelerator optimized for serving a suite of workloads improves Perf/TDP by 2.4× on average compared to TPU-v3. Our return on investment analysis shows that FAST-generated accelerators can potentially be practical for moderate-sized datacenter deployments. @InProceedings{ASPLOS22p27, author = {Dan Zhang and Safeen Huda and Ebrahim Songhori and Kartik Prabhu and Quoc Le and Anna Goldie and Azalia Mirhoseini}, title = {A Full-Stack Search Technique for Domain Optimized Deep Learning Accelerators}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {27--42}, doi = {10.1145/3503222.3507767}, year = {2022}, } Publisher's Version
	Huici, Felipe	ASPLOS '22: "FlexOS: Towards Flexible OS ..." FlexOS: Towards Flexible OS Isolation Hugo Lefeuvre, Vlad-Andrei Bădoiu, Alexander Jung, Stefan Lucian Teodorescu, Sebastian Rauch, Felipe Huici, Costin Raiciu, and Pierre Olivier (University of Manchester, UK; University Politehnica of Bucharest, Romania; Lancaster University, UK; unikraft.io, Germany; KIT, Germany; NEC Laboratories, Germany; Correct Networks, Romania) At design time, modern operating systems are locked in a specific safety and isolation strategy that mixes one or more hardware/software protection mechanisms (e.g. user/kernel separation); revisiting these choices after deployment requires a major refactoring effort. This rigid approach shows its limits given the wide variety of modern applications' safety/performance requirements, when new hardware isolation mechanisms are rolled out, or when existing ones break. We present FlexOS, a novel OS allowing users to easily specialize the safety and isolation strategy of an OS at compilation/deployment time instead of design time. This modular LibOS is composed of fine-grained components that can be isolated via a range of hardware protection mechanisms with various data sharing strategies and additional software hardening. The OS ships with an exploration technique helping the user navigate the vast safety/performance design space it unlocks. We implement a prototype of the system and demonstrate, for several applications (Redis/Nginx/SQLite), FlexOS' vast configuration space as well as the efficiency of the exploration technique: we evaluate 80 FlexOS configurations for Redis and show how that space can be probabilistically subset to the 5 safest ones under a given performance budget. We also show that, under equivalent configurations, FlexOS performs similarly or better than existing solutions which use fixed safety configurations. @InProceedings{ASPLOS22p467, author = {Hugo Lefeuvre and Vlad-Andrei Bădoiu and Alexander Jung and Stefan Lucian Teodorescu and Sebastian Rauch and Felipe Huici and Costin Raiciu and Pierre Olivier}, title = {FlexOS: Towards Flexible OS Isolation}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {467--482}, doi = {10.1145/3503222.3507759}, year = {2022}, } Publisher's Version Published Artifact Info Artifacts Available Artifacts Functional Results Reproduced
	Iancu, Costin	ASPLOS '22: "QUEST: Systematically Approximating ..." QUEST: Systematically Approximating Quantum Circuits for Higher Output Fidelity Tirthak Patel, Ed Younis, Costin Iancu, Wibe de Jong, and Devesh Tiwari (Northeastern University, USA; Lawrence Berkeley National Laboratory, USA) We present QUEST, a procedure to systematically generate approximations for quantum circuits to reduce their CNOT gate count. Our approach employs circuit partitioning for scalability with procedures to 1) reduce circuit length using approximate synthesis, 2) improve fidelity by running circuits that represent key samples in the approximation space, and 3) reason about approximation upper bound. Our evaluation results indicate that our approach of "dissimilar" approximations provides close fidelity to the original circuit. Overall, the results indicate that QUEST can reduce CNOT gate count by 30-80% on ideal systems and decrease the impact of noise on existing and near-future quantum systems. @InProceedings{ASPLOS22p514, author = {Tirthak Patel and Ed Younis and Costin Iancu and Wibe de Jong and Devesh Tiwari}, title = {QUEST: Systematically Approximating Quantum Circuits for Higher Output Fidelity}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {514--528}, doi = {10.1145/3503222.3507739}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional
	Iyigun, Mehmet	ASPLOS '22: "Memory-Harvesting VMs in Cloud ..." Memory-Harvesting VMs in Cloud Platforms Alexander Fuerst, Stanko Novaković, Íñigo Goiri, Gohar Irfan Chaudhry, Prateek Sharma, Kapil Arya, Kevin Broas, Eugene Bak, Mehmet Iyigun, and Ricardo Bianchini (Indiana University, USA; Microsoft Research, USA; Microsoft Azure, USA) loud platforms monetize their spare capacity by renting “Spot” virtual machines (VMs) that can be evicted in favor of higher-priority VMs. Recent work has shown that resource-harvesting VMs are more effective at exploiting spare capacity than Spot VMs, while also reducing the number of evictions. However, the prior work focused on harvesting CPU cores while keeping memory size fixed. This wastes a substantial monetization opportunity and may even limit the ability of harvesting VMs to leverage spare cores. Thus, in this paper, we explore memory harvesting and its challenges in real cloud platforms, namely its impact on VM creation time, NUMA spanning, and page fragmentation. We start by characterizing the amount and dynamics of the spare memory in Azure. We then design and implement memory-harvesting VMs (MHVMs), introducing new techniques for memory buffering, batching, and pre-reclamation. To demonstrate the use of MHVMs, we also extend a popular cluster scheduling framework (Hadoop) and a FaaS platform to adapt to them. Our main results show that (1) there is plenty of scope for memory harvesting in real platforms; (2) MHVMs are effective at mitigating the negative impacts of harvesting; and (3) our extensions of Hadoop and FaaS successfully hide the MHVMs’ varying memory size from the users’ data-processing jobs and functions. We conclude that memory harvesting has great potential for practical deployment and users can save up to 93% of their costs when running workloads on MHVMs. @InProceedings{ASPLOS22p583, author = {Alexander Fuerst and Stanko Novaković and Íñigo Goiri and Gohar Irfan Chaudhry and Prateek Sharma and Kapil Arya and Kevin Broas and Eugene Bak and Mehmet Iyigun and Ricardo Bianchini}, title = {Memory-Harvesting VMs in Cloud Platforms}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {583--594}, doi = {10.1145/3503222.3507725}, year = {2022}, } Publisher's Version
	Jain, Mayank	ASPLOS '22: "TMO: Transparent Memory Offloading ..." TMO: Transparent Memory Offloading in Datacenters Johannes Weiner, Niket Agarwal, Dan Schatzberg, Leon Yang, Hao Wang, Blaise Sanouillet, Bikash Sharma, Tejun Heo, Mayank Jain, Chunqiang Tang, and Dimitrios Skarlatos (Meta, USA; Carnegie Mellon University, USA) The unrelenting growth of the memory needs of emerging datacenter applications, along with ever increasing cost and volatility of DRAM prices, has led to DRAM being a major infrastructure expense. Alternative technologies, such as NVMe SSDs and upcoming NVM devices, offer higher capacity than DRAM at a fraction of the cost and power. One promising approach is to transparently offload colder memory to cheaper memory technologies via kernel or hypervisor techniques. The key challenge, however, is to develop a datacenter-scale solution that is robust in dealing with diverse workloads and large performance variance of different offload devices such as compressed memory, SSD, and NVM. This paper presents TMO, Meta’s transparent memory offloading solution for heterogeneous datacenter environments. TMO introduces a new Linux kernel mechanism that directly measures in realtime the lost work due to resource shortage across CPU, memory, and I/O. Guided by this information and without any prior application knowledge, TMO automatically adjusts how much memory to offload to heterogeneous devices (e.g., compressed memory or SSD) according to the device’s performance characteristics and the application’s sensitivity to memory-access slowdown. TMO holistically identifies offloading opportunities from not only the application containers but also the sidecar containers that provide infrastructure-level functions. To maximize memory savings, TMO targets both anonymous memory and file cache, and balances the swap-in rate of anonymous memory and the reload rate of file pages that were recently evicted from the file cache. TMO has been running in production for more than a year, and has saved between 20-32% of the total memory across millions of servers in our large datacenter fleet. We have successfully upstreamed TMO into the Linux kernel. @InProceedings{ASPLOS22p609, author = {Johannes Weiner and Niket Agarwal and Dan Schatzberg and Leon Yang and Hao Wang and Blaise Sanouillet and Bikash Sharma and Tejun Heo and Mayank Jain and Chunqiang Tang and Dimitrios Skarlatos}, title = {TMO: Transparent Memory Offloading in Datacenters}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {609--621}, doi = {10.1145/3503222.3507731}, year = {2022}, } Publisher's Version
	Jangda, Abhinav	ASPLOS '22: "Breaking the Computation and ..." Breaking the Computation and Communication Abstraction Barrier in Distributed Machine Learning Workloads Abhinav Jangda, Jun Huang, Guodong Liu, Amir Hossein Nodehi Sabet, Saeed Maleki, Youshan Miao, Madanlal Musuvathi, Todd Mytkowicz, and Olli Saarikivi (University of Massachusetts at Amherst, USA; Ohio State University, USA; Chinese Academy of Sciences, China; University of California at Riverside, USA; Microsoft Research, USA; Microsoft Research, China) Recent trends towards large machine learning models require both training and inference tasks to be distributed. Considering the huge cost of training these models, it is imperative to unlock optimizations in computation and communication to obtain best performance. However, the current logical separation between computation and communication kernels in machine learning frameworks misses optimization opportunities across this barrier. Breaking this abstraction can provide many optimizations to improve the performance of distributed workloads. However, manually applying these optimizations requires modifying the underlying computation and communication libraries for each scenario, which is both time consuming and error-prone. Therefore, we present CoCoNet, which contains (i) a domain specific language to express a distributed machine learning program in the form of computation and communication operations, (ii) a set of semantics preserving transformations to optimize the program, and (iii) a compiler to generate jointly optimized communication and computation GPU kernels. Providing both computation and communication as first class constructs allows users to work on a high-level abstraction and apply powerful optimizations, such as fusion or overlapping of communication and computation. CoCoNet enabled us to optimize data-, model- and pipeline-parallel workloads in large language models with only a few lines of code. Our experiments show that CoCoNet significantly outperforms state-of-the-art distributed machine learning implementations. @InProceedings{ASPLOS22p402, author = {Abhinav Jangda and Jun Huang and Guodong Liu and Amir Hossein Nodehi Sabet and Saeed Maleki and Youshan Miao and Madanlal Musuvathi and Todd Mytkowicz and Olli Saarikivi}, title = {Breaking the Computation and Communication Abstraction Barrier in Distributed Machine Learning Workloads}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {402--416}, doi = {10.1145/3503222.3507778}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional
	Javadi-Abhari, Ali	ASPLOS '22: "Paulihedral: A Generalized ..." Paulihedral: A Generalized Block-Wise Compiler Optimization Framework for Quantum Simulation Kernels Gushu Li, Anbang Wu, Yunong Shi, Ali Javadi-Abhari, Yufei Ding, and Yuan Xie (University of California at Santa Barbara, USA; Amazon, USA; IBM, USA) The quantum simulation kernel is an important subroutine appearing as a very long gate sequence in many quantum programs. In this paper, we propose Paulihedral, a block-wise compiler framework that can deeply optimize this subroutine by exploiting high-level program structure and optimization opportunities. Paulihedral first employs a new Pauli intermediate representation that can maintain the high-level semantics and constraints in quantum simulation kernels. This naturally enables new large-scale optimizations that are hard to implement at the low gate-level. In particular, we propose two technology-independent instruction scheduling passes, and two technology-dependent code optimization passes which reconcile the circuit synthesis, gate cancellation, and qubit mapping stages of the compiler. Experimental results show that Paulihedral can outperform state-of-the-art compiler infrastructures in a wide-range of applications on both near-term superconducting quantum processors and future fault-tolerant quantum computers. @InProceedings{ASPLOS22p554, author = {Gushu Li and Anbang Wu and Yunong Shi and Ali Javadi-Abhari and Yufei Ding and Yuan Xie}, title = {Paulihedral: A Generalized Block-Wise Compiler Optimization Framework for Quantum Simulation Kernels}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {554--569}, doi = {10.1145/3503222.3507715}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Ji, Houxiang	ASPLOS '22: "Pinned Loads: Taming Speculative ..." Pinned Loads: Taming Speculative Loads in Secure Processors Zirui Neil Zhao, Houxiang Ji, Adam Morrison, Darko Marinov, and Josep Torrellas (University of Illinois at Urbana-Champaign, USA; Tel Aviv University, Israel) In security frameworks for speculative execution, an instruction is said to reach its Visibility Point (VP) when it is no longer vulnerable to pipeline squashes. Before a potentially leaky instruction reaches its VP, it has to stall—unless a defense scheme such as invisible speculation provides protection. Unfortunately, either stalling or protecting the execution of pre-VP instructions typically has a performance cost. One way to attain low-overhead safe execution is to develop techniques that speed-up the advance of the VP from older to younger instructions. In this paper, we propose one such technique. We find that the progress of the VP for loads is mostly impeded by waiting until no memory consistency violations (MCVs) are possible. Hence, our technique, called , tries to make loads invulnerable to MCVs as early as possible—a process we call pinning the loads in the pipeline. The result is faster VP progress and a reduction in the execution overhead of defense schemes. In this paper, we describe the hardware needed by , and two possible designs with different tradeoffs between hardware requirements and performance. Our evaluation shows that is very effective: extending three popular defense schemes against speculative execution attacks with reduces their average execution overhead on SPEC17 and on SPLASH2/PARSEC applications by about 50%. For example, on SPEC17, the execution overhead of the three defense schemes decreases from to , from to , and from to . @InProceedings{ASPLOS22p314, author = {Zirui Neil Zhao and Houxiang Ji and Adam Morrison and Darko Marinov and Josep Torrellas}, title = {Pinned Loads: Taming Speculative Loads in Secure Processors}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {314--328}, doi = {10.1145/3503222.3507724}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Jia, Ziyang	ASPLOS '22: "CoolEdge: Hotspot-Relievable ..." CoolEdge: Hotspot-Relievable Warm Water Cooling for Energy-Efficient Edge Datacenters Qiangyu Pei, Shutong Chen, Qixia Zhang, Xinhui Zhu, Fangming Liu, Ziyang Jia, Yishuo Wang, and Yongjie Yuan (Huazhong University of Science and Technology, China) As the computing frontier drifts to the edge, edge datacenters play a crucial role in supporting various real-time applications. Different from cloud datacenters, the requirements of proximity to end-users, high density, and heterogeneity, present new challenges to cool the edge datacenters efficiently. Although warm water cooling has become a promising cooling technique for this infrastructure, the one-size-fits-all cooling control would lower the cooling efficiency considerably because of the severe thermal imbalance across servers, hardware, and even inside one hardware component in an edge datacenter. In this work, we propose CoolEdge, a hotspot-relievable warm water cooling system for improving the cooling efficiency and saving costs of edge datacenters. Specifically, through the elaborate design of water circulations, CoolEdge can dynamically adjust the water temperature and flow rate for each heterogeneous hardware component to eliminate the hardware-level hotspots. By redesigning cold plates, CoolEdge can quickly disperse the chip-level hotspots without manual intervention. We further quantify the power saving achieved by the warm water cooling theoretically, and propose a custom-designed cooling solution to decide an appropriate water temperature and flow rate periodically. Based on a hardware prototype and real-world traces from SURFsara, the evaluation results show that CoolEdge reduces the cooling energy by 81.81% and 71.92%, respectively, compared with conventional and state-of-the-art water cooling systems. @InProceedings{ASPLOS22p814, author = {Qiangyu Pei and Shutong Chen and Qixia Zhang and Xinhui Zhu and Fangming Liu and Ziyang Jia and Yishuo Wang and Yongjie Yuan}, title = {CoolEdge: Hotspot-Relievable Warm Water Cooling for Energy-Efficient Edge Datacenters}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {814--829}, doi = {10.1145/3503222.3507713}, year = {2022}, } Publisher's Version
	Jiang, Lin	ASPLOS '22: "JSONSki: Streaming Semi-structured ..." JSONSki: Streaming Semi-structured Data with Bit-Parallel Fast-Forwarding Lin Jiang and Zhijia Zhao (University of California at Riverside, USA) Semi-structured data, such as JSON, are fundamental to the Web and document data stores. Streaming analytics on semi-structured data combines parsing and query evaluation into one pass to avoid generating parse trees. Though promising, its conventional design requires to parse the data stream in detail character by character, which limits the efficiency of streaming analytics. This work reveals a wide range of opportunities to fast-forward the streaming over certain data substructures irrelevant to the query evaluation. However, identifying these substructures itself may need detailed parsing. To resolve this dilemma, this work designs a highly bit-parallel solution that intensively utilizes bitwise and SIMD operations to identify the irrelevant substructures during the streaming. It includes a new streaming model—recursive-descent streaming, for an easy adoption of fast-forward optimizations, a concept—structural intervals, for partitioning the data stream, and a group of bit-parallel algorithms implementing various fast-forward cases. The solution is implemented as a JSON streaming framework, called JSONSki. It offers a set of APIs that can be invoked during the streaming to dynamically fast-forward over different cases of irrelevant substructures. Evaluation using real-world datasets and standard path queries shows that JSONSki can achieve significant speedups over the state-of-the-art JSON processing tools while taking a minimum memory footprint. @InProceedings{ASPLOS22p200, author = {Lin Jiang and Zhijia Zhao}, title = {JSONSki: Streaming Semi-structured Data with Bit-Parallel Fast-Forwarding}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {200--211}, doi = {10.1145/3503222.3507719}, year = {2022}, } Publisher's Version
	Jiang, Muhui	ASPLOS '22: "EXAMINER: Automatically Locating ..." EXAMINER: Automatically Locating Inconsistent Instructions between Real Devices and CPU Emulators for ARM Muhui Jiang, Tianyi Xu, Yajin Zhou, Yufeng Hu, Ming Zhong, Lei Wu, Xiapu Luo, and Kui Ren (Hong Kong Polytechnic University, China; Zhejiang University, China) Emulators are widely used to build dynamic analysis frameworks due to its fine-grained tracing capability, full system monitoring functionality, and scalability of running on different operating systems and architectures. However, whether emulators are consistent with real devices is unknown. To understand this problem, we aim to automatically locate inconsistent instructions, which behave differently between emulators and real devices. We target the ARM architecture, which provides machine-readable specifications. Based on the specification, we propose a sufficient test case generator by designing and implementing the first symbolic execution engine for the ARM architecture specification language (ASL). We generate 2,774,649 representative instruction streams and conduct differential testing between four ARM real devices in different architecture versions (i.e., ARMv5, ARMv6, ARMv7, and ARMv8) and three state-of-the-art emulators (i.e., QEMU, Unicorn, and Angr). We locate a huge number of inconsistent instruction streams (171,858 for QEMU, 223,264 for unicorn, and 120,169 for Angr). We find that undefined implementation in ARM manual and bugs of emulators are the major causes of inconsistencies. Furthermore, we discover 12 bugs, which influence commonly used instructions (e.g., BLX). With the inconsistent instructions, we build three security applications and demonstrate the capability of these instructions on detecting emulators, anti-emulation, and anti-fuzzing. @InProceedings{ASPLOS22p846, author = {Muhui Jiang and Tianyi Xu and Yajin Zhou and Yufeng Hu and Ming Zhong and Lei Wu and Xiapu Luo and Kui Ren}, title = {EXAMINER: Automatically Locating Inconsistent Instructions between Real Devices and CPU Emulators for ARM}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {846--858}, doi = {10.1145/3503222.3507736}, year = {2022}, } Publisher's Version
	Jiang, Xueqiang	ASPLOS '22: "Serverless Computing on Heterogeneous ..." Serverless Computing on Heterogeneous Computers Dong Du, Qingyuan Liu, Xueqiang Jiang, Yubin Xia, Binyu Zang, and Haibo Chen (Shanghai Jiao Tong University, China; Shanghai Artificial Intelligence Laboratory, China) Existing serverless computing platforms are built upon homogeneous computers, limiting the function density and restricting serverless computing to limited scenarios. We introduce Molecule, the first serverless computing system utilizing heterogeneous computers. Molecule enables both general-purpose devices (e.g., Nvidia DPU) and domain-specific accelerators (e.g., FPGA and GPU) for serverless applications that significantly improve function density (50% higher) and application performance (up to 34.6x). To achieve these results, we first propose XPU-Shim, a distributed shim to bridge the gap between underlying multi-OS systems (when using general-purpose devices) and our serverless runtime (i.e., Molecule). We further introduce vectorized sandbox, a sandbox abstraction to abstract hardware heterogeneity (when using domain-specific accelerators). Moreover, we also review state-of-the-art serverless optimizations on startup and communication latency and overcome the challenges to implement them on heterogeneous computers. We have implemented Molecule on real platforms with Nvidia DPUs and Xilinx FPGAs and evaluate it using benchmarks and real-world applications. @InProceedings{ASPLOS22p797, author = {Dong Du and Qingyuan Liu and Xueqiang Jiang and Yubin Xia and Binyu Zang and Haibo Chen}, title = {Serverless Computing on Heterogeneous Computers}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {797--813}, doi = {10.1145/3503222.3507732}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Jones, Cody	ASPLOS '22: "LILLIPUT: A Lightweight Low-Latency ..." LILLIPUT: A Lightweight Low-Latency Lookup-Table Decoder for Near-Term Quantum Error Correction Poulami Das, Aditya Locharla, and Cody Jones (Georgia Institute of Technology, USA; Google, USA) The error rates of quantum devices are orders of magnitude higher than what is needed to run most quantum applications. To close this gap, Quantum Error Correction (QEC) encodes logical qubits and distributes information using several physical qubits. By periodically executing a syndrome extraction circuit on the logical qubits, information about errors (called syndrome) is extracted while running programs. A decoder uses these syndromes to identify and correct errors in real time, which is necessary to prevent accumulation of errors. Unfortunately, software decoders are slow and hardware decoders are fast but less accurate. Thus, almost all QEC studies so far have relied on offline decoding. To enable real-time decoding in near-term QEC, we propose LILLIPUT– a Lightweight Low Latency Look-Up Table decoder. LILLIPUT consists of two parts– First, it translates syndromes into error detection events that index into a Look-Up Table (LUT) whose entry provides the error information in real-time. Second, it programs the LUTs with error assignments for all possible error events by running a software decoder offline. LILLIPUT tolerates an error on any operation in the quantum hardware, including gates and measurements, and the number of tolerated errors grows with the size of the code. LILLIPUT utilizes less than 7% logic on off-the-shelf FPGAs enabling practical adoption, as FPGAs are already used to design the control and readout circuits in existing systems. LILLIPUT incurs a latency of a few nanoseconds and enables real-time decoding. We also propose Compressed LUTs (CLUTs) to reduce the memory required by LILLIPUT. By exploiting the fact that not all error events are equally likely and only storing data for the most probable error events, CLUTs reduce the memory needed by up-to 107x (from 148 MB to 1.38 MB) without degrading the accuracy. @InProceedings{ASPLOS22p541, author = {Poulami Das and Aditya Locharla and Cody Jones}, title = {LILLIPUT: A Lightweight Low-Latency Lookup-Table Decoder for Near-Term Quantum Error Correction}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {541--553}, doi = {10.1145/3503222.3507707}, year = {2022}, } Publisher's Version
	Jones, Timothy M.	ASPLOS '22: "MineSweeper: A “Clean Sweep” ..." MineSweeper: A “Clean Sweep” for Drop-In Use-after-Free Prevention Márton Erdős, Sam Ainsworth, and Timothy M. Jones (University of Cambridge, UK; University of Edinburgh, UK) Low-level languages, which require manual memory management from the programmer, remain in wide use for performance-critical applications. Memory-safety bugs are common, and now a major source of exploits. In particular, a use-after-free bug occurs when an object is erroneously deallocated, whilst pointers to it remain active in memory, and those (dangling) pointers are later used to access the object. An attacker can reallocate the memory area backing an erroneously freed object, then overwrite its contents, injecting carefully chosen data into the host program, thus altering its execution and achieving privilege escalation. We present MineSweeper, a system to mitigate use-after-free vulnerabilities by retaining freed allocations in a quarantine, until no pointers to them remain in program memory, thus preventing their reallocation until it is safe. MineSweeper performs efficient linear sweeps of memory to identify quarantined items that have no dangling pointers to them, and thus can be safely reallocated. This allows MineSweeper to be significantly more efficient than previous transitive marking procedure techniques. MineSweeper, attached to JeMalloc, improves security at an acceptable overhead in memory footprint (11.1% on average) and an execution-time cost of only 5.4% (geometric mean for SPEC CPU2006), with 9.6% additional threaded CPU usage. These figures considerably improve on the state-of-the-art for non-probabilistic drop-in temporal-safety systems, and make MineSweeper the only such scheme suitable for deployment in real-world production environments. @InProceedings{ASPLOS22p212, author = {Márton Erdős and Sam Ainsworth and Timothy M. Jones}, title = {MineSweeper: A “Clean Sweep” for Drop-In Use-after-Free Prevention}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {212--225}, doi = {10.1145/3503222.3507712}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Jung, Alexander	ASPLOS '22: "FlexOS: Towards Flexible OS ..." FlexOS: Towards Flexible OS Isolation Hugo Lefeuvre, Vlad-Andrei Bădoiu, Alexander Jung, Stefan Lucian Teodorescu, Sebastian Rauch, Felipe Huici, Costin Raiciu, and Pierre Olivier (University of Manchester, UK; University Politehnica of Bucharest, Romania; Lancaster University, UK; unikraft.io, Germany; KIT, Germany; NEC Laboratories, Germany; Correct Networks, Romania) At design time, modern operating systems are locked in a specific safety and isolation strategy that mixes one or more hardware/software protection mechanisms (e.g. user/kernel separation); revisiting these choices after deployment requires a major refactoring effort. This rigid approach shows its limits given the wide variety of modern applications' safety/performance requirements, when new hardware isolation mechanisms are rolled out, or when existing ones break. We present FlexOS, a novel OS allowing users to easily specialize the safety and isolation strategy of an OS at compilation/deployment time instead of design time. This modular LibOS is composed of fine-grained components that can be isolated via a range of hardware protection mechanisms with various data sharing strategies and additional software hardening. The OS ships with an exploration technique helping the user navigate the vast safety/performance design space it unlocks. We implement a prototype of the system and demonstrate, for several applications (Redis/Nginx/SQLite), FlexOS' vast configuration space as well as the efficiency of the exploration technique: we evaluate 80 FlexOS configurations for Redis and show how that space can be probabilistically subset to the 5 safest ones under a given performance budget. We also show that, under equivalent configurations, FlexOS performs similarly or better than existing solutions which use fixed safety configurations. @InProceedings{ASPLOS22p467, author = {Hugo Lefeuvre and Vlad-Andrei Bădoiu and Alexander Jung and Stefan Lucian Teodorescu and Sebastian Rauch and Felipe Huici and Costin Raiciu and Pierre Olivier}, title = {FlexOS: Towards Flexible OS Isolation}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {467--482}, doi = {10.1145/3503222.3507759}, year = {2022}, } Publisher's Version Published Artifact Info Artifacts Available Artifacts Functional Results Reproduced
	Kamath, Aditya K	ASPLOS '22: "GPM: Leveraging Persistent ..." GPM: Leveraging Persistent Memory from a GPU Shweta Pandey, Aditya K Kamath, and Arkaprava Basu (IISc Bangalore, India) The GPU is a key computing platform for many application domains. While the new non-volatile memory technology has brought the promise of byte-addressable persistence (a.k.a., persistent memory, or PM) to CPU applications, the same, unfortunately, is beyond the reach of GPU programs. We take three key steps toward enabling GPU programs to access PM directly. First, enable direct access to PM from within a GPU kernel without needing to modify the hardware. Next, we demonstrate three classes of GPU-accelerated applications that benefit from PM. In the process, we create a workload suite with nine such applications. We then create a GPU library, written in CUDA, to support logging, checkpointing, and primitives for native persistence for programmers to easily leverage PM. @InProceedings{ASPLOS22p142, author = {Shweta Pandey and Aditya K Kamath and Arkaprava Basu}, title = {GPM: Leveraging Persistent Memory from a GPU}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {142--156}, doi = {10.1145/3503222.3507758}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Kamil, Shoaib	ASPLOS '22: "Vector Instruction Selection ..." Vector Instruction Selection for Digital Signal Processors using Program Synthesis Maaz Bin Safeer Ahmad, Alexander J. Root, Andrew Adams, Shoaib Kamil, and Alvin Cheung (Adobe, USA; Massachusetts Institute of Technology, USA; University of California at Berkeley, USA) Instruction selection, whereby input code represented in an intermediate representation is translated into executable instructions from the target platform, is often the most target-dependent component in optimizing compilers. Current approaches include pattern matching, which is brittle and tedious to design, or search-based methods, which are limited by scalability of the search algorithm. In this paper, we propose a new algorithm that first abstracts the target platform instructions into high-level uber-instructions, with each uber-instruction unifying multiple concrete instructions from the target platform. Program synthesis is used to lift input code sequences into semantically equivalent sequences of uber-instructions and then to lower from uber-instructions to machine code. Using 21 real-world benchmarks, we show that our synthesis-based instruction selection algorithm can generate instruction sequences for a hardware target, with the synthesized code performing up to 2.1x faster as compared to code generated by a professionally-developed optimizing compiler for the same platform. @InProceedings{ASPLOS22p1004, author = {Maaz Bin Safeer Ahmad and Alexander J. Root and Andrew Adams and Shoaib Kamil and Alvin Cheung}, title = {Vector Instruction Selection for Digital Signal Processors using Program Synthesis}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1004--1016}, doi = {10.1145/3503222.3507714}, year = {2022}, } Publisher's Version
	Kannan, Tejas	ASPLOS '22: "Protecting Adaptive Sampling ..." Protecting Adaptive Sampling from Information Leakage on Low-Power Sensors Tejas Kannan and Henry Hoffmann (University of Chicago, USA) Adaptive sampling is a powerful family of algorithms for managing energy consumption on low-power sensors. These algorithms use captured measurements to control the sensor's collection rate, leading to near-optimal error under energy constraints. Adaptive sampling's data-driven nature, however, comes at a cost in privacy. In this work, we demonstrate how the collection rates of general adaptive policies leak information about captured measurements. Further, individual adaptive policies display this leakage on multiple tasks. This result presents a challenge in maintaining privacy for sensors using energy-efficient batched communication. In this context, the size of measurement batches exposes the sampling policy's collection rate. Thus, an attacker who monitors the encrypted link between sensor and server can use message lengths to uncover information about the captured values. We address this side-channel by introducing a framework called Adaptive Group Encoding (AGE) that protects any periodic adaptive sampler. AGE uses quantization to encode all batches as fixed-length messages, making message sizes independent of the collection rate. AGE reduces the quantization error through a series of transformations. The proposed framework preserves the low error of adaptive sampling while preventing information leakage and incurring negligible energy overhead. @InProceedings{ASPLOS22p240, author = {Tejas Kannan and Henry Hoffmann}, title = {Protecting Adaptive Sampling from Information Leakage on Low-Power Sensors}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {240--254}, doi = {10.1145/3503222.3507775}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Kasikci, Baris	ASPLOS '22: "Debugging in the Brave New ..." Debugging in the Brave New World of Reconfigurable Hardware Jiacheng Ma, Gefei Zuo, Kevin Loughlin, Haoyang Zhang, Andrew Quinn, and Baris Kasikci (University of Michigan, USA; University of California at Santa Cruz, USA) Software and hardware development cycles have traditionally been quite distinct. Software allows post-deployment patches, which leads to a rapid development cycle. In contrast, hardware bugs that are found after fabrication are extremely costly to fix (and sometimes even unfixable), so the traditional hardware development cycle involves massive investment in extensive simulation and formal verification. Reconfigurable hardware, such as a Field Programmable Gate Array (FPGA), promises to propel hardware development towards an agile software-like development approach, since it enables a hardware developer to patch bugs that are detected during on-chip testing or in production. Unfortunately, FPGA programmers lack bug localization tools amenable to this rapid development cycle, since past tools mainly find bugs via simulation and verification. To develop hardware bug localization tools for a rapid development cycle, a thorough understanding of the symptoms, root causes, and fixes of hardware bugs is needed. In this paper, we first study bugs in existing FPGA designs and produce a testbed of reliably-reproducible bugs. We classify the bugs according to their intrinsic properties, symptoms, and root causes. We demonstrate that many hardware bugs are comparable to software bug counterparts, and would benefit from similar techniques for bug diagnosis and repair. Based upon our findings, we build a novel collection of hybrid static/dynamic program analysis and monitoring tools for debugging FPGA designs, showing that our tools enable a software-like development cycle by effectively reducing developers' manual efforts for bug localization. @InProceedings{ASPLOS22p946, author = {Jiacheng Ma and Gefei Zuo and Kevin Loughlin and Haoyang Zhang and Andrew Quinn and Baris Kasikci}, title = {Debugging in the Brave New World of Reconfigurable Hardware}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {946--962}, doi = {10.1145/3503222.3507701}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Kersnar, Drew	ASPLOS '22: "CARAT CAKE: Replacing Paging ..." CARAT CAKE: Replacing Paging via Compiler/Kernel Cooperation Brian Suchy, Souradip Ghosh, Drew Kersnar, Siyuan Chai, Zhen Huang, Aaron Nelson, Michael Cuevas, Alex Bernat, Gaurav Chaudhary, Nikos Hardavellas, Simone Campanoni, and Peter Dinda (Northwestern University, USA) Virtual memory, specifically paging, is undergoing significant innovation due to being challenged by new demands from modern workloads. Recent work has demonstrated an alternative software only design that can result in simplified hardware requirements, even supporting purely physical addressing. While we have made the case for this Compiler- And Runtime-based Address Translation (CARAT) concept, its evaluation was based on a user-level prototype. We now report on incorporating CARAT into a kernel, forming Compiler- And Runtime-based Address Translation for CollAborative Kernel Environments (CARAT CAKE). In our implementation, a Linux-compatible x64 process abstraction can be based either on CARAT CAKE, or on a sophisticated paging implementation. Implementing CARAT CAKE involves kernel changes and compiler optimizations/transformations that must work on all code in the system, including kernel code. We evaluate CARAT CAKE in comparison with paging and find that CARAT CAKE is able to achieve the functionality of paging (protection, mapping, and movement properties) with minimal overhead. In turn, CARAT CAKE allows significant new benefits for systems including energy savings, larger L1 caches, and arbitrary granularity memory management. @InProceedings{ASPLOS22p98, author = {Brian Suchy and Souradip Ghosh and Drew Kersnar and Siyuan Chai and Zhen Huang and Aaron Nelson and Michael Cuevas and Alex Bernat and Gaurav Chaudhary and Nikos Hardavellas and Simone Campanoni and Peter Dinda}, title = {CARAT CAKE: Replacing Paging via Compiler/Kernel Cooperation}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {98--114}, doi = {10.1145/3503222.3507771}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available
	Kim, Jangwoo	ASPLOS '22: "CryoWire: Wire-Driven Microarchitecture ..." CryoWire: Wire-Driven Microarchitecture Designs for Cryogenic Computing Dongmoon Min, Yujin Chung, Ilkwon Byun, Junpyo Kim, and Jangwoo Kim (Seoul National University, South Korea) Cryogenic computing, which runs a computer device at an extremely low temperature, is promising thanks to its significant reduction of wire resistance as well as leakage current. Recent studies on cryogenic computing have focused on various architectural units including the main memory, cache, and CPU core running at 77K. However, little research has been conducted to fully exploit the fast cryogenic wires, even though the slow wires are becoming more serious performance bottleneck in modern processors. In this paper, we propose a CPU microarchitecture which extensively exploits the fast wires at 77K. For this goal, we first introduce our validated cryogenic-performance models for the CPU pipeline and network on chip (NoC), whose performance can be significantly limited by the slow wires. Next, based on the analysis with the models, we architect CryoSP and CryoBus as our pipeline and NoC designs to fully exploit the fast wires. Our evaluation shows that our cryogenic computer equipped with both microarchitectures achieves 3.82 times higher system-level performance compared to the conventional computer system thanks to the 96% higher clock frequency of CryoSP and five times lower NoC latency of CryoBus. @InProceedings{ASPLOS22p903, author = {Dongmoon Min and Yujin Chung and Ilkwon Byun and Junpyo Kim and Jangwoo Kim}, title = {CryoWire: Wire-Driven Microarchitecture Designs for Cryogenic Computing}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {903--917}, doi = {10.1145/3503222.3507749}, year = {2022}, } Publisher's Version
	Kim, Jeremie	ASPLOS '22: "GenStore: A High-Performance ..." GenStore: A High-Performance In-Storage Processing System for Genome Sequence Analysis Nika Mansouri Ghiasi, Jisung Park, Harun Mustafa, Jeremie Kim, Ataberk Olgun, Arvid Gollwitzer, Damla Senol Cali, Can Firtina, Haiyu Mao, Nour Almadhoun Alserr, Rachata Ausavarungnirun, Nandita Vijaykumar, Mohammed Alser, and Onur Mutlu (ETH Zurich, Switzerland; Bionano Genomics, USA; King Mongkut's University of Technology North Bangkok, Thailand; University of Toronto, Canada) Read mapping is a fundamental step in many genomics applications. It is used to identify potential matches and differences between fragments (called reads) of a sequenced genome and an already known genome (called a reference genome). Read mapping is costly because it needs to perform approximate string matching (ASM) on large amounts of data. To address the computational challenges in genome analysis, many prior works propose various approaches such as accurate filters that select the reads within a dataset of genomic reads (called a read set) that must undergo expensive computation, efficient heuristics, and hardware acceleration. While effective at reducing the amount of expensive computation, all such approaches still require the costly movement of a large amount of data from storage to the rest of the system, which can significantly lower the end-to-end performance of read mapping in conventional and emerging genomics systems. We propose GenStore, the first in-storage processing system designed for genome sequence analysis that greatly reduces both data movement and computational overheads of genome sequence analysis by exploiting low-cost and accurate in-storage filters. GenStore leverages hardware/software co-design to address the challenges of in-storage processing, supporting reads with 1) different properties such as read lengths and error rates, which highly depend on the sequencing technology, and 2) different degrees of genetic variation compared to the reference genome, which highly depends on the genomes that are being compared. Through rigorous analysis of read mapping processes of reads with different properties and degrees of genetic variation, we meticulously design low-cost hardware accelerators and data/computation flows inside a NAND flash-based solid-state drive (SSD). Our evaluation using a wide range of real genomic datasets shows that GenStore, when implemented in three modern NAND flash-based SSDs, significantly improves the read mapping performance of state-of-the-art software (hardware) baselines by 2.07-6.05× (1.52-3.32×) for read sets with high similarity to the reference genome and 1.45-33.63× (2.70-19.2×) for read sets with low similarity to the reference genome. @InProceedings{ASPLOS22p635, author = {Nika Mansouri Ghiasi and Jisung Park and Harun Mustafa and Jeremie Kim and Ataberk Olgun and Arvid Gollwitzer and Damla Senol Cali and Can Firtina and Haiyu Mao and Nour Almadhoun Alserr and Rachata Ausavarungnirun and Nandita Vijaykumar and Mohammed Alser and Onur Mutlu}, title = {GenStore: A High-Performance In-Storage Processing System for Genome Sequence Analysis}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {635--654}, doi = {10.1145/3503222.3507702}, year = {2022}, } Publisher's Version
	Kim, Junpyo	ASPLOS '22: "CryoWire: Wire-Driven Microarchitecture ..." CryoWire: Wire-Driven Microarchitecture Designs for Cryogenic Computing Dongmoon Min, Yujin Chung, Ilkwon Byun, Junpyo Kim, and Jangwoo Kim (Seoul National University, South Korea) Cryogenic computing, which runs a computer device at an extremely low temperature, is promising thanks to its significant reduction of wire resistance as well as leakage current. Recent studies on cryogenic computing have focused on various architectural units including the main memory, cache, and CPU core running at 77K. However, little research has been conducted to fully exploit the fast cryogenic wires, even though the slow wires are becoming more serious performance bottleneck in modern processors. In this paper, we propose a CPU microarchitecture which extensively exploits the fast wires at 77K. For this goal, we first introduce our validated cryogenic-performance models for the CPU pipeline and network on chip (NoC), whose performance can be significantly limited by the slow wires. Next, based on the analysis with the models, we architect CryoSP and CryoBus as our pipeline and NoC designs to fully exploit the fast wires. Our evaluation shows that our cryogenic computer equipped with both microarchitectures achieves 3.82 times higher system-level performance compared to the conventional computer system thanks to the 96% higher clock frequency of CryoSP and five times lower NoC latency of CryoBus. @InProceedings{ASPLOS22p903, author = {Dongmoon Min and Yujin Chung and Ilkwon Byun and Junpyo Kim and Jangwoo Kim}, title = {CryoWire: Wire-Driven Microarchitecture Designs for Cryogenic Computing}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {903--917}, doi = {10.1145/3503222.3507749}, year = {2022}, } Publisher's Version
	Kim, Miryung	ASPLOS '22: "HeteroGen: Transpiling C to ..." HeteroGen: Transpiling C to Heterogeneous HLS Code with Automated Test Generation and Program Repair Qian Zhang, Jiyuan Wang, Guoqing Harry Xu, and Miryung Kim (University of California at Los Angeles, USA) Despite the trend of incorporating heterogeneity and specialization in hardware, the development of heterogeneous applications is limited to a handful of engineers with deep hardware expertise. We propose HeteroGen that takes C/C++ code as input and automatically generates an HLS version with test behavior preservation and better performance. Key to the success of HeteroGen is adapting the idea of search-based program repair to the heterogeneous computing domain, while addressing two technical challenges. First, the turn-around time of HLS compilation and simulation is much longer than the usual C/C++ compilation and execution time; therefore, HeteroGen applies pattern-oriented program edits guided by common fix patterns and their dependences. Second, behavior and performance checking requires testing, but test cases are often unavailable. Thus, HeteroGen auto-generates test inputs suitable for checking C to HLS-C conversion errors, while providing high branch coverage for the original C code. An evaluation of HeteroGen shows that it produces an HLS-compatible version for nine out of ten real-world heterogeneous applications fully automatically, applying up to 438 lines of edits to produce an HLS version 1.63x faster than the original version. @InProceedings{ASPLOS22p1017, author = {Qian Zhang and Jiyuan Wang and Guoqing Harry Xu and Miryung Kim}, title = {HeteroGen: Transpiling C to Heterogeneous HLS Code with Automated Test Generation and Program Repair}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1017--1029}, doi = {10.1145/3503222.3507748}, year = {2022}, } Publisher's Version Published Artifact Info Artifacts Available Artifacts Functional Results Reproduced
	Kokolis, Apostolos	ASPLOS '22: "Parallel Virtualized Memory ..." Parallel Virtualized Memory Translation with Nested Elastic Cuckoo Page Tables Jovan Stojkovic, Dimitrios Skarlatos, Apostolos Kokolis, Tianyin Xu, and Josep Torrellas (University of Illinois at Urbana-Champaign, USA; Carnegie Mellon University, USA) A major reason why nested or virtualized address translations are slow is because current systems organize page tables in a multi-level tree that is accessed in a sequential manner. A nested translation may potentially require up to twenty-four sequential memory accesses. To address this problem, this paper presents the first page table design that supports parallel nested address translation. The design is based on using hashed page tables (HPTs) for both guest and host. However, directly extending a native HPT design to a nested environment leads to minor gains. Instead, our design solves a new set of challenges that appear in nested environments. Our scheme eliminates all but three of the potentially twenty-four sequential steps of a nested translation—while judiciously limiting the number of parallel memory accesses issued to avoid over-consuming cache bandwidth. As a result, compared to conventional nested radix tables, our design speeds-up the execution of a set of applications by an average of 1.19x (for 4KB pages) and 1.24x (when huge pages are used). In addition, we also show a migration path from current nested radix page tables to our design. @InProceedings{ASPLOS22p84, author = {Jovan Stojkovic and Dimitrios Skarlatos and Apostolos Kokolis and Tianyin Xu and Josep Torrellas}, title = {Parallel Virtualized Memory Translation with Nested Elastic Cuckoo Page Tables}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {84--97}, doi = {10.1145/3503222.3507720}, year = {2022}, } Publisher's Version
	Köpf, Boris	ASPLOS '22: "Revizor: Testing Black-Box ..." Revizor: Testing Black-Box CPUs against Speculation Contracts Oleksii Oleksenko, Christof Fetzer, Boris Köpf, and Mark Silberstein (TU Dresden, Germany; Microsoft Research, UK; Technion, Israel) Speculative vulnerabilities such as Spectre and Meltdown expose speculative execution state that can be exploited to leak information across security domains via side-channels. Such vulnerabilities often stay undetected for a long time as we lack the tools for systematic testing of CPUs to find them. In this paper, we propose an approach to automatically detect microarchitectural information leakage in commercial black-box CPUs. We build on speculation contracts, which we employ to specify the permitted side effects of program execution on the CPU's microarchitectural state. We propose a Model-based Relational Testing (MRT) technique to empirically assess the CPU compliance with these specifications. We implement MRT in a testing framework called Revizor, and showcase its effectiveness on real Intel x86 CPUs. Revizor automatically detects violations of a rich set of contracts, or indicates their absence. A highlight of our findings is that Revizor managed to automatically surface Spectre, MDS, and LVI, as well as several previously unknown variants. @InProceedings{ASPLOS22p226, author = {Oleksii Oleksenko and Christof Fetzer and Boris Köpf and Mark Silberstein}, title = {Revizor: Testing Black-Box CPUs against Speculation Contracts}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {226--239}, doi = {10.1145/3503222.3507729}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Korolija, Dario	ASPLOS '22: "Enzian: An Open, General, ..." Enzian: An Open, General, CPU/FPGA Platform for Systems Software Research David Cock, Abishek Ramdas, Daniel Schwyn, Michael Giardino, Adam Turowski, Zhenhao He, Nora Hossle, Dario Korolija, Melissa Licciardello, Kristina Martsenko, Reto Achermann, Gustavo Alonso, and Timothy Roscoe (ETH Zurich, Switzerland; University of British Columbia, Canada) Hybrid computing platforms, comprising CPU cores and FPGA logic, are increasingly used for accelerating data-intensive workloads in cloud deployments, and are a growing topic of interest in systems research. However, from a research perspective, existing hardware platforms are limited: they are often optimized for concrete, narrow use-cases and, therefore lack the flexibility needed to explore other applications and configurations. We show that a research group can design and build a more general, open, and affordable hardware platform for hybrid systems research. The platform, Enzian, is capable of duplicating the functionality of existing CPU/FPGA systems with comparable performance but in an open, flexible system. It couples a large FPGA with a server-class CPU in an asymmetric cache-coherent NUMA system. Enzian also enables research not possible with existing hybrid platforms, through explicit access to coherence messages, extensive thermal and power instrumentation, and an open, programmable baseboard management processor. Enzian is already being used in multiple projects, is open source (both hardware and software), and available for remote use. We present the design principles of Enzian, the challenges in building it, and evaluate it with a range of existing research use-cases alongside other, more specialized platforms, as well as demonstrating research not possible on existing platforms. @InProceedings{ASPLOS22p434, author = {David Cock and Abishek Ramdas and Daniel Schwyn and Michael Giardino and Adam Turowski and Zhenhao He and Nora Hossle and Dario Korolija and Melissa Licciardello and Kristina Martsenko and Reto Achermann and Gustavo Alonso and Timothy Roscoe}, title = {Enzian: An Open, General, CPU/FPGA Platform for Systems Software Research}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {434--451}, doi = {10.1145/3503222.3507742}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Kozyrakis, Christos	ASPLOS '22: "ShEF: Shielded Enclaves for ..." ShEF: Shielded Enclaves for Cloud FPGAs Mark Zhao, Mingyu Gao, and Christos Kozyrakis (Stanford University, USA; Tsinghua University, China) FPGAs are now used in public clouds to accelerate a wide range of applications, including many that operate on sensitive data such as financial and medical records. We present ShEF, a trusted execution environment (TEE) for cloud-based reconfigurable accelerators. ShEF is independent from CPU-based TEEs and allows secure execution under a threat model where the adversary can control all software running on the CPU connected to the FPGA, has physical access to the FPGA, and can compromise the FPGA interface logic of the cloud provider. ShEF provides a secure boot and remote attestation process that relies solely on existing FPGA mechanisms for root of trust. It also includes a Shield component that provides secure access to data while the accelerator is in use. The Shield is highly customizable and extensible, allowing users to craft a bespoke security solution that fits their accelerator's memory access patterns, bandwidth, and security requirements at minimum performance and area overheads. We describe a prototype implementation of ShEF for existing cloud FPGAs, map ShEF to a performant and secure storage application, and measure the performance benefits of customizable security using five additional accelerators. @InProceedings{ASPLOS22p1070, author = {Mark Zhao and Mingyu Gao and Christos Kozyrakis}, title = {ShEF: Shielded Enclaves for Cloud FPGAs}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1070--1085}, doi = {10.1145/3503222.3507733}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional ASPLOS '22: "RecShard: Statistical Feature-Based ..." RecShard: Statistical Feature-Based Memory Optimization for Industry-Scale Neural Recommendation Geet Sethi, Bilge Acun, Niket Agarwal, Christos Kozyrakis, Caroline Trippel, and Carole-Jean Wu (Stanford University, USA; Meta, USA) We propose RecShard, a fine-grained embedding table (EMB) partitioning and placement technique for deep learning recommendation models (DLRMs). RecShard is designed based on two key observations. First, not all EMBs are equal, nor all rows within an EMB are equal in terms of access patterns. EMBs exhibit distinct memory characteristics, providing performance optimization opportunities for intelligent EMB partitioning and placement across a tiered memory hierarchy. Second, in modern DLRMs, EMBs function as hash tables. As a result, EMBs display interesting phenomena, such as the birthday paradox, leaving EMBs severely under-utilized. RecShard determines an optimal EMB sharding strategy for a set of EMBs based on training data distributions and model characteristics, along with the bandwidth characteristics of the underlying tiered memory hierarchy. In doing so, RecShard achieves over 6 times higher EMB training throughput on average for capacity constrained DLRMs. The throughput increase comes from improved EMB load balance by over 12 times and from the reduced access to the slower memory by over 87 times. @InProceedings{ASPLOS22p344, author = {Geet Sethi and Bilge Acun and Niket Agarwal and Christos Kozyrakis and Caroline Trippel and Carole-Jean Wu}, title = {RecShard: Statistical Feature-Based Memory Optimization for Industry-Scale Neural Recommendation}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {344--358}, doi = {10.1145/3503222.3507777}, year = {2022}, } Publisher's Version ASPLOS '22: "SOL: Safe On-Node Learning ..." SOL: Safe On-Node Learning in Cloud Platforms Yawen Wang, Daniel Crankshaw, Neeraja J. Yadwadkar, Daniel Berger, Christos Kozyrakis, and Ricardo Bianchini (Stanford University, USA; Microsoft Research, USA; University of Texas at Austin, USA) Cloud platforms run many software agents on each server node. These agents manage all aspects of node operation, and in some cases frequently collect data and make decisions. Unfortunately, their behavior is typically based on pre-defined static heuristics or offline analysis; they do not leverage on-node machine learning (ML). In this paper, we first characterize the spectrum of node agents in Azure, and identify the classes of agents that are most likely to benefit from on-node ML. We then propose SOL, an extensible framework for designing ML-based agents that are safe and robust to the range of failure conditions that occur in production. SOL provides a simple API to agent developers and manages the scheduling and running of the agent-specific functions they write. We illustrate the use of SOL by implementing three ML-based agents that manage CPU cores, node power, and memory placement. Our experiments show that (1) ML substantially improves our agents, and (2) SOL ensures that agents operate safely under a variety of failure conditions. We conclude that ML-based agents show significant potential and that SOL can help build them. @InProceedings{ASPLOS22p622, author = {Yawen Wang and Daniel Crankshaw and Neeraja J. Yadwadkar and Daniel Berger and Christos Kozyrakis and Ricardo Bianchini}, title = {SOL: Safe On-Node Learning in Cloud Platforms}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {622--634}, doi = {10.1145/3503222.3507704}, year = {2022}, } Publisher's Version
	Lazo, Cristóbal Ramírez	ASPLOS '22: "BiSon-e: A Lightweight and ..." BiSon-e: A Lightweight and High-Performance Accelerator for Narrow Integer Linear Algebra Computing on the Edge Enrico Reggiani, Cristóbal Ramírez Lazo, Roger Figueras Bagué, Adrián Cristal, Mauro Olivieri, and Osman Sabri Unsal (Polytechnic University of Catalonia, Spain; Barcelona Supercomputing Center, Spain; Sapienza University of Rome, Italy) Linear algebra computational kernels based on byte and sub-byte integer data formats are at the base of many classes of applications, ranging from Deep Learning to Pattern Matching. Porting the computation of these applications from cloud to edge and mobile devices would enable significant improvements in terms of security, safety, and energy efficiency. However, despite their low memory and energy demands, their intrinsically high computational intensity makes the execution of these workloads challenging on highly resource-constrained devices. In this paper, we present BiSon-e, a novel RISC-V based architecture that accelerates linear algebra kernels based on narrow integer computations on edge processors by performing Single Instruction Multiple Data (SIMD) operations on off-the-shelf scalar Functional Units (FUs). Our novel architecture is built upon the binary segmentation technique, which allows to significantly reduce the memory footprint and the arithmetic intensity of linear algebra kernels requiring narrow data sizes. We integrate BiSon-e into a complete System-on-Chip (SoC) based on RISC-V, synthesized and Place&Routed in 65nm and 22nm technologies, introducing a negligible 0.07% area overhead with respect to the baseline architecture. Our experimental evaluation shows that, when computing the Convolution and Fully-Connected layers of the AlexNet and VGG-16 Convolutional Neural Networks (CNNs) with 8-, 4-, and 2-bit, our solution gains up to 5.6×, 13.9× and 24× in execution time compared to the scalar implementation of a single RISC-V core, and improves the energy efficiency of string matching tasks by 5× when compared to a RISC-V-based Vector Processing Unit (VPU). @InProceedings{ASPLOS22p56, author = {Enrico Reggiani and Cristóbal Ramírez Lazo and Roger Figueras Bagué and Adrián Cristal and Mauro Olivieri and Osman Sabri Unsal}, title = {BiSon-e: A Lightweight and High-Performance Accelerator for Narrow Integer Linear Algebra Computing on the Edge}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {56--69}, doi = {10.1145/3503222.3507746}, year = {2022}, } Publisher's Version
	Le, Quoc	ASPLOS '22: "A Full-Stack Search Technique ..." A Full-Stack Search Technique for Domain Optimized Deep Learning Accelerators Dan Zhang, Safeen Huda, Ebrahim Songhori, Kartik Prabhu, Quoc Le, Anna Goldie, and Azalia Mirhoseini (Google Brain, USA; Google, USA; Stanford University, USA) The rapidly-changing deep learning landscape presents a unique opportunity for building inference accelerators optimized for specific datacenter-scale workloads. We propose Full-stack Accelerator Search Technique (FAST), a hardware accelerator search framework that defines a broad optimization environment covering key design decisions within the hardware-software stack, including hardware datapath, software scheduling, and compiler passes such as operation fusion and tensor padding. In this paper, we analyze bottlenecks in state-of-the-art vision and natural language processing (NLP) models, including EfficientNet and BERT, and use FAST to design accelerators capable of addressing these bottlenecks. FAST-generated accelerators optimized for single workloads improve Perf/TDP by 3.7× on average across all benchmarks compared to TPU-v3. A FAST-generated accelerator optimized for serving a suite of workloads improves Perf/TDP by 2.4× on average compared to TPU-v3. Our return on investment analysis shows that FAST-generated accelerators can potentially be practical for moderate-sized datacenter deployments. @InProceedings{ASPLOS22p27, author = {Dan Zhang and Safeen Huda and Ebrahim Songhori and Kartik Prabhu and Quoc Le and Anna Goldie and Azalia Mirhoseini}, title = {A Full-Stack Search Technique for Domain Optimized Deep Learning Accelerators}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {27--42}, doi = {10.1145/3503222.3507767}, year = {2022}, } Publisher's Version
	Lefeuvre, Hugo	ASPLOS '22: "FlexOS: Towards Flexible OS ..." FlexOS: Towards Flexible OS Isolation Hugo Lefeuvre, Vlad-Andrei Bădoiu, Alexander Jung, Stefan Lucian Teodorescu, Sebastian Rauch, Felipe Huici, Costin Raiciu, and Pierre Olivier (University of Manchester, UK; University Politehnica of Bucharest, Romania; Lancaster University, UK; unikraft.io, Germany; KIT, Germany; NEC Laboratories, Germany; Correct Networks, Romania) At design time, modern operating systems are locked in a specific safety and isolation strategy that mixes one or more hardware/software protection mechanisms (e.g. user/kernel separation); revisiting these choices after deployment requires a major refactoring effort. This rigid approach shows its limits given the wide variety of modern applications' safety/performance requirements, when new hardware isolation mechanisms are rolled out, or when existing ones break. We present FlexOS, a novel OS allowing users to easily specialize the safety and isolation strategy of an OS at compilation/deployment time instead of design time. This modular LibOS is composed of fine-grained components that can be isolated via a range of hardware protection mechanisms with various data sharing strategies and additional software hardening. The OS ships with an exploration technique helping the user navigate the vast safety/performance design space it unlocks. We implement a prototype of the system and demonstrate, for several applications (Redis/Nginx/SQLite), FlexOS' vast configuration space as well as the efficiency of the exploration technique: we evaluate 80 FlexOS configurations for Redis and show how that space can be probabilistically subset to the 5 safest ones under a given performance budget. We also show that, under equivalent configurations, FlexOS performs similarly or better than existing solutions which use fixed safety configurations. @InProceedings{ASPLOS22p467, author = {Hugo Lefeuvre and Vlad-Andrei Bădoiu and Alexander Jung and Stefan Lucian Teodorescu and Sebastian Rauch and Felipe Huici and Costin Raiciu and Pierre Olivier}, title = {FlexOS: Towards Flexible OS Isolation}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {467--482}, doi = {10.1145/3503222.3507759}, year = {2022}, } Publisher's Version Published Artifact Info Artifacts Available Artifacts Functional Results Reproduced
	Lei, Yu	ASPLOS '22: "One Size Does Not Fit All: ..." One Size Does Not Fit All: Security Hardening of MIPS Embedded Systems via Static Binary Debloating for Shared Libraries Haotian Zhang, Mengfei Ren, Yu Lei, and Jiang Ming (University of Texas at Arlington, USA) Embedded systems have become prominent targets for cyberattacks. To exploit firmware’s memory corruption vulnerabilities, cybercriminals harvest reusable code gadgets from the large shared library codebase (e.g., uClibc). Unfortunately, unlike their desktop counterparts, embedded systems lack essential computing resources to enforce security hardening techniques. Recently, we have witnessed a surge of software debloating as a new defense mechanism against code-reuse attacks; it erases unused code to significantly diminish the possibilities of constructing reusable gadgets. Because of the single firmware image update style, static library debloating shows promise to fortify embedded systems without compromising performance and forward compatibility. However, static library debloating on stripped binaries (e.g., firmware’s shared libraries) is still an enormous challenge. In this paper, we show that this challenge is not insurmountable for MIPS firmware. We develop a novel system, named uTrimmer, to identify and wipe out unused basic blocks from shared libraries’ binary code, without causing additional runtime overhead or memory consumption. We propose a new method to identify address-taken blocks/functions, which further help us maintain an inter-procedural control flow graph to conservatively include library code that could be potentially used by firmware. By capturing address access patterns for position-independent code, we circumvent the challenge of determining code-pointer targets and safely eliminate unused code. We run uTrimmer to debloat shared libraries for SPEC CPU2017 benchmarks, popular firmware applications (e.g., Apache, BusyBox, and OpenSSL), and a real-world wireless router firmware image. Our experiments show that not only does uTrimmer deliver functional programs, but also it can cut the exposed code surface and eliminate various reusable code gadgets remarkably. uTrimmer’s debloating capability can compete with the static linking results. @InProceedings{ASPLOS22p255, author = {Haotian Zhang and Mengfei Ren and Yu Lei and Jiang Ming}, title = {One Size Does Not Fit All: Security Hardening of MIPS Embedded Systems via Static Binary Debloating for Shared Libraries}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {255--270}, doi = {10.1145/3503222.3507768}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Leng, Jingwen	ASPLOS '22: "VELTAIR: Towards High-Performance ..." VELTAIR: Towards High-Performance Multi-tenant Deep Learning Services via Adaptive Compilation and Scheduling Zihan Liu, Jingwen Leng, Zhihui Zhang, Quan Chen, Chao Li, and Minyi Guo (Shanghai Jiao Tong University, China; Shanghai Qi Zhi Institute, China) Deep learning (DL) models have achieved great success in many application domains. As such, many industrial companies such as Google and Facebook have acknowledged the importance of multi-tenant DL services. Although the multi-tenant service has been studied in conventional workloads, it is not been deeply studied on deep learning service, especially on general-purpose hardware. In this work, we systematically analyze the opportunities and challenges of providing multi-tenant deep learning services on the general-purpose CPU architecture from the aspects of scheduling granularity and code generation. We propose an adaptive granularity scheduling scheme to both guarantee resource usage efficiency and reduce the scheduling conflict rate. We also propose an adaptive compilation strategy, by which we can dynamically and intelligently pick a program with proper exclusive and shared resource usage to reduce overall interference-induced performance loss. Compared to the existing works, our design can serve more requests under the same QoS target in various scenarios (e.g., +71%, +62%, +45% for light, medium, and heavy workloads, respectively), and reduce the averaged query latency by 50%. @InProceedings{ASPLOS22p388, author = {Zihan Liu and Jingwen Leng and Zhihui Zhang and Quan Chen and Chao Li and Minyi Guo}, title = {VELTAIR: Towards High-Performance Multi-tenant Deep Learning Services via Adaptive Compilation and Scheduling}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {388--401}, doi = {10.1145/3503222.3507752}, year = {2022}, } Publisher's Version ASPLOS '22: "Astraea: Towards QoS-Aware ..." Astraea: Towards QoS-Aware and Resource-Efficient Multi-stage GPU Services Wei Zhang, Quan Chen, Kaihua Fu, Ningxin Zheng, Zhiyi Huang, Jingwen Leng, and Minyi Guo (Shanghai Jiao Tong University, China; Microsoft Research, China; University of Otago, New Zealand) Multi-stage user-facing applications on GPUs are widely-used nowa- days, and are often implemented to be microservices. Prior re- search works are not applicable to ensuring QoS of GPU-based microservices due to the different communication patterns and shared resource contentions. We propose Astraea to manage GPU microservices considering the above factors. In Astraea, a microser- vice deployment policy is used to maximize the supported peak service load while ensuring the required QoS. To adaptively switch the communication methods between microservices according to different deployments, we propose an auto-scaling GPU communi- cation framework. The framework automatically scales based on the currently used hardware topology and microservice location, and adopts global memory-based techniques to reduce intra-GPU communication. Astraea increases the supported peak load by up to 82.3% while achieving the desired 99%-ile latency target compared with state-of-the-art solutions. @InProceedings{ASPLOS22p570, author = {Wei Zhang and Quan Chen and Kaihua Fu and Ningxin Zheng and Zhiyi Huang and Jingwen Leng and Minyi Guo}, title = {Astraea: Towards QoS-Aware and Resource-Efficient Multi-stage GPU Services}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {570--582}, doi = {10.1145/3503222.3507721}, year = {2022}, } Publisher's Version
	Levi, Dotan	ASPLOS '22: "FlexDriver: A Network Driver ..." FlexDriver: A Network Driver for Your Accelerator Haggai Eran, Maxim Fudim, Gabi Malka, Gal Shalom, Noam Cohen, Amit Hermony, Dotan Levi, Liran Liss, and Mark Silberstein (NVIDIA, Israel; Technion, Israel) We propose a new system design for connecting hardware and FPGA accelerators to the network, allowing the accelerator to directly control commodity Network Interface Cards (NICs) without using the CPU. This enables us to solve the key challenge of leveraging existing NIC hardware offloads such as virtualization, tunneling, and RDMA for accelerator networking. Our approach supports a diverse set of use cases, from direct network access for disaggregated accelerators to inline-acceleration of the network stack, all without the complex networking logic in the accelerator. To demonstrate the feasibility of this approach, we build FlexDriver (FLD), an on-accelerator hardware module that implements a NIC data-plane driver. Our main technical contribution is a mechanism that compresses the NIC control structures by two orders of magnitude, allowing FLD to achieve high networking scalability with low die area cost and no bandwidth interference with the accelerator logic. The prototype for NVIDIA Innova-2 FPGA SmartNICs showcases our design’s utility for three different accelerators: a disaggregated LTE cipher, an IP-defragmentation inline accelerator, and an IoT cryptographic-token authentication offload. These accelerators reach 25 Gbps line rate and leverage the NIC for RDMA processing, VXLAN tunneling, and traffic shaping without CPU involvement. @InProceedings{ASPLOS22p1115, author = {Haggai Eran and Maxim Fudim and Gabi Malka and Gal Shalom and Noam Cohen and Amit Hermony and Dotan Levi and Liran Liss and Mark Silberstein}, title = {FlexDriver: A Network Driver for Your Accelerator}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1115--1129}, doi = {10.1145/3503222.3507776}, year = {2022}, } Publisher's Version
	Li, Chao	ASPLOS '22: "VELTAIR: Towards High-Performance ..." VELTAIR: Towards High-Performance Multi-tenant Deep Learning Services via Adaptive Compilation and Scheduling Zihan Liu, Jingwen Leng, Zhihui Zhang, Quan Chen, Chao Li, and Minyi Guo (Shanghai Jiao Tong University, China; Shanghai Qi Zhi Institute, China) Deep learning (DL) models have achieved great success in many application domains. As such, many industrial companies such as Google and Facebook have acknowledged the importance of multi-tenant DL services. Although the multi-tenant service has been studied in conventional workloads, it is not been deeply studied on deep learning service, especially on general-purpose hardware. In this work, we systematically analyze the opportunities and challenges of providing multi-tenant deep learning services on the general-purpose CPU architecture from the aspects of scheduling granularity and code generation. We propose an adaptive granularity scheduling scheme to both guarantee resource usage efficiency and reduce the scheduling conflict rate. We also propose an adaptive compilation strategy, by which we can dynamically and intelligently pick a program with proper exclusive and shared resource usage to reduce overall interference-induced performance loss. Compared to the existing works, our design can serve more requests under the same QoS target in various scenarios (e.g., +71%, +62%, +45% for light, medium, and heavy workloads, respectively), and reduce the averaged query latency by 50%. @InProceedings{ASPLOS22p388, author = {Zihan Liu and Jingwen Leng and Zhihui Zhang and Quan Chen and Chao Li and Minyi Guo}, title = {VELTAIR: Towards High-Performance Multi-tenant Deep Learning Services via Adaptive Compilation and Scheduling}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {388--401}, doi = {10.1145/3503222.3507752}, year = {2022}, } Publisher's Version
	Li, Cheng	ASPLOS '22: "NASPipe: High Performance ..." NASPipe: High Performance and Reproducible Pipeline Parallel Supernet Training via Causal Synchronous Parallelism Shixiong Zhao, Fanxin Li, Xusheng Chen, Tianxiang Shen, Li Chen, Sen Wang, Nicholas Zhang, Cheng Li, and Heming Cui (University of Hong Kong, China; Huawei Technologies, China; University of Science and Technology of China, China) Supernet training, a prevalent and important paradigm in Neural Architecture Search, embeds the whole DNN architecture search space into one monolithic supernet, iteratively activates a subset of the supernet (i.e., a subnet) for fitting each batch of data, and searches a high-quality subnet which meets specific requirements. Although training subnets in parallel on multiple GPUs is desirable for acceleration, there inherently exists a race hazard that concurrent subnets may access the same DNN layers. Existing systems support neither efficiently parallelizing subnets’ training executions, nor resolving the race hazard deterministically, leading to unreproducible training procedures and potentiallly non-trivial accuracy loss. We present NASPipe, the first high-performance and reproducible distributed supernet training system via causal synchronous parallel (CSP) pipeline scheduling abstraction: NASPipe partitions a supernet across GPUs and concurrently executes multiple generated sub-tasks (subnets) in a pipelined manner; meanwhile, it oversees the correlations between the subnets and deterministically resolves any causal dependency caused by subnets’ layer sharing. To obtain high performance, NASPipe’s CSP scheduler exploits the fact that the larger a supernet spans, the fewer dependencies manifest between chronologically close subnets; therefore, it aggressively schedules the subnets with larger chronological orders into execution, only if they are not causally dependent on unfinished precedent subnets. Moreover, to relieve the excessive GPU memory burden for holding the whole supernet’s parameters, NASPipe uses a context switch technique that stashes the whole supernet in CPU memory, precisely predicts the subnets’ schedule, and pre-fetches/evicts a subnet before/after its execution. The evaluation shows that NASPipe is the only system that retains supernet training reproducibility, while achieving a comparable and even higher performance (up to 7.8X) compared to three recent pipeline training systems (e.g., GPipe). @InProceedings{ASPLOS22p374, author = {Shixiong Zhao and Fanxin Li and Xusheng Chen and Tianxiang Shen and Li Chen and Sen Wang and Nicholas Zhang and Cheng Li and Heming Cui}, title = {NASPipe: High Performance and Reproducible Pipeline Parallel Supernet Training via Causal Synchronous Parallelism}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {374--387}, doi = {10.1145/3503222.3507735}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Li, Fanxin	ASPLOS '22: "NASPipe: High Performance ..." NASPipe: High Performance and Reproducible Pipeline Parallel Supernet Training via Causal Synchronous Parallelism Shixiong Zhao, Fanxin Li, Xusheng Chen, Tianxiang Shen, Li Chen, Sen Wang, Nicholas Zhang, Cheng Li, and Heming Cui (University of Hong Kong, China; Huawei Technologies, China; University of Science and Technology of China, China) Supernet training, a prevalent and important paradigm in Neural Architecture Search, embeds the whole DNN architecture search space into one monolithic supernet, iteratively activates a subset of the supernet (i.e., a subnet) for fitting each batch of data, and searches a high-quality subnet which meets specific requirements. Although training subnets in parallel on multiple GPUs is desirable for acceleration, there inherently exists a race hazard that concurrent subnets may access the same DNN layers. Existing systems support neither efficiently parallelizing subnets’ training executions, nor resolving the race hazard deterministically, leading to unreproducible training procedures and potentiallly non-trivial accuracy loss. We present NASPipe, the first high-performance and reproducible distributed supernet training system via causal synchronous parallel (CSP) pipeline scheduling abstraction: NASPipe partitions a supernet across GPUs and concurrently executes multiple generated sub-tasks (subnets) in a pipelined manner; meanwhile, it oversees the correlations between the subnets and deterministically resolves any causal dependency caused by subnets’ layer sharing. To obtain high performance, NASPipe’s CSP scheduler exploits the fact that the larger a supernet spans, the fewer dependencies manifest between chronologically close subnets; therefore, it aggressively schedules the subnets with larger chronological orders into execution, only if they are not causally dependent on unfinished precedent subnets. Moreover, to relieve the excessive GPU memory burden for holding the whole supernet’s parameters, NASPipe uses a context switch technique that stashes the whole supernet in CPU memory, precisely predicts the subnets’ schedule, and pre-fetches/evicts a subnet before/after its execution. The evaluation shows that NASPipe is the only system that retains supernet training reproducibility, while achieving a comparable and even higher performance (up to 7.8X) compared to three recent pipeline training systems (e.g., GPipe). @InProceedings{ASPLOS22p374, author = {Shixiong Zhao and Fanxin Li and Xusheng Chen and Tianxiang Shen and Li Chen and Sen Wang and Nicholas Zhang and Cheng Li and Heming Cui}, title = {NASPipe: High Performance and Reproducible Pipeline Parallel Supernet Training via Causal Synchronous Parallelism}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {374--387}, doi = {10.1145/3503222.3507735}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Li, Gushu	ASPLOS '22: "Paulihedral: A Generalized ..." Paulihedral: A Generalized Block-Wise Compiler Optimization Framework for Quantum Simulation Kernels Gushu Li, Anbang Wu, Yunong Shi, Ali Javadi-Abhari, Yufei Ding, and Yuan Xie (University of California at Santa Barbara, USA; Amazon, USA; IBM, USA) The quantum simulation kernel is an important subroutine appearing as a very long gate sequence in many quantum programs. In this paper, we propose Paulihedral, a block-wise compiler framework that can deeply optimize this subroutine by exploiting high-level program structure and optimization opportunities. Paulihedral first employs a new Pauli intermediate representation that can maintain the high-level semantics and constraints in quantum simulation kernels. This naturally enables new large-scale optimizations that are hard to implement at the low gate-level. In particular, we propose two technology-independent instruction scheduling passes, and two technology-dependent code optimization passes which reconcile the circuit synthesis, gate cancellation, and qubit mapping stages of the compiler. Experimental results show that Paulihedral can outperform state-of-the-art compiler infrastructures in a wide-range of applications on both near-term superconducting quantum processors and future fault-tolerant quantum computers. @InProceedings{ASPLOS22p554, author = {Gushu Li and Anbang Wu and Yunong Shi and Ali Javadi-Abhari and Yufei Ding and Yuan Xie}, title = {Paulihedral: A Generalized Block-Wise Compiler Optimization Framework for Quantum Simulation Kernels}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {554--569}, doi = {10.1145/3503222.3507715}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Li, Jie	ASPLOS '22: "INFless: A Native Serverless ..." INFless: A Native Serverless System for Low-Latency, High-Throughput Inference Yanan Yang, Laiping Zhao, Yiming Li, Huanyu Zhang, Jie Li, Mingyang Zhao, Xingzhen Chen, and Keqiu Li (Tianjin University, China; 58.com, China) Modern websites increasingly rely on machine learning (ML) to improve their business efficiency. Developing and maintaining ML services incurs high costs for developers. Although serverless systems are a promising solution to reduce costs, we find that the current general purpose serverless systems cannot meet the low latency, high throughput demands of ML services. While simply ”patching” general serverless systems does not resolve the problem completely, we propose that such a system should natively combine the features of inference with a serverless paradigm. We present INFless, the first ML domain-specific serverless platform. It provides a unified, heterogeneous resource abstraction between CPU and accelerators, and achieves high throughput using built-in batching and non-uniform scaling mechanisms. It also supports low latency through coordinated management of batch queuing time, execution time and coldstart rate. We evaluate INFless using both a local cluster testbed and a large-scale simulation. Experimental results show that INFless outperforms state-of-the-art systems by 2×-5× on system throughput, meeting the latency goals of ML services. @InProceedings{ASPLOS22p768, author = {Yanan Yang and Laiping Zhao and Yiming Li and Huanyu Zhang and Jie Li and Mingyang Zhao and Xingzhen Chen and Keqiu Li}, title = {INFless: A Native Serverless System for Low-Latency, High-Throughput Inference}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {768--781}, doi = {10.1145/3503222.3507709}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available
	Li, Jing (Jane)	ASPLOS '22: "Software-Defined Address Mapping: ..." Software-Defined Address Mapping: A Case on 3D Memory Jialiang Zhang, Michael Swift, and Jing (Jane) Li (University of Pennsylvania, USA; University of Wisconsin-Madison, USA) 3D-stacking memory such as High-Bandwidth Memory (HBM) and Hybrid Memory Cube (HMC) provides orders of magnitude more bandwidth and significantly increased channel-level parallelism (CLP) due to its new parallel memory architecture. However, it is challenging to fully exploit the abundant CLP for performance as the bandwidth utilization is highly dependent on address mapping in the memory controller. Unfortunately, CLP is very sensitive to a program’s data access pattern, which is not made available to OS/hardware by existing mechanisms. In this work, we address these challenges with software-defined address mapping (SDAM) that, for the first time, enables user program to obtain a direct control of the low-level memory hardware in a more intelligent and fine-grained manner. In particular, we develop new mechanisms that can effectively communicate a program’s data access properties to the OS and hardware and to use it to control data placement in hardware. To guarantee correctness and reduce overhead in storage and performance, we extend Linux kernel and C-language memory allocators to support multiple address mappings. For advanced system optimization, we develop machine learning methods that can automatically identify access patterns of major variables in a program and cluster these with similar access patterns to reduce the overhead for SDAM. We demonstrate the benefits of our design on real system prototype, comprising (1) a RISC-V processor, near memory accelerators and HBM modules using Xilinx FPGA platform, and (2) modified Linux and glibc. Our evaluation on standard CPU benchmarks and data-intensive benchmarks (for both CPU and accelerators) demonstrates 1.41×, 1.84× speedup on CPU and 2.58× on near memory accelerators in our system with SDAM compared to a baseline system that uses a fixed address mapping. @InProceedings{ASPLOS22p70, author = {Jialiang Zhang and Michael Swift and Jing (Jane) Li}, title = {Software-Defined Address Mapping: A Case on 3D Memory}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {70--83}, doi = {10.1145/3503222.3507774}, year = {2022}, } Publisher's Version
	Li, Keqiu	ASPLOS '22: "INFless: A Native Serverless ..." INFless: A Native Serverless System for Low-Latency, High-Throughput Inference Yanan Yang, Laiping Zhao, Yiming Li, Huanyu Zhang, Jie Li, Mingyang Zhao, Xingzhen Chen, and Keqiu Li (Tianjin University, China; 58.com, China) Modern websites increasingly rely on machine learning (ML) to improve their business efficiency. Developing and maintaining ML services incurs high costs for developers. Although serverless systems are a promising solution to reduce costs, we find that the current general purpose serverless systems cannot meet the low latency, high throughput demands of ML services. While simply ”patching” general serverless systems does not resolve the problem completely, we propose that such a system should natively combine the features of inference with a serverless paradigm. We present INFless, the first ML domain-specific serverless platform. It provides a unified, heterogeneous resource abstraction between CPU and accelerators, and achieves high throughput using built-in batching and non-uniform scaling mechanisms. It also supports low latency through coordinated management of batch queuing time, execution time and coldstart rate. We evaluate INFless using both a local cluster testbed and a large-scale simulation. Experimental results show that INFless outperforms state-of-the-art systems by 2×-5× on system throughput, meeting the latency goals of ML services. @InProceedings{ASPLOS22p768, author = {Yanan Yang and Laiping Zhao and Yiming Li and Huanyu Zhang and Jie Li and Mingyang Zhao and Xingzhen Chen and Keqiu Li}, title = {INFless: A Native Serverless System for Low-Latency, High-Throughput Inference}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {768--781}, doi = {10.1145/3503222.3507709}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available
	Li, Tuo	ASPLOS '22: "Path-Sensitive and Alias-Aware ..." Path-Sensitive and Alias-Aware Typestate Analysis for Detecting OS Bugs Tuo Li, Jia-Ju Bai, Yulei Sui, and Shi-Min Hu (Tsinghua University, China; University of Technology Sydney, Australia) Operating system (OS) is the cornerstone for modern computer systems. It manages devices and provides fundamental service for user-level applications. Thus, detecting bugs in OSes is important to improve reliability and security of computer systems. Static typestate analysis is a common technique for detecting different types of bugs, but it is often inaccurate or unscalable for large-size OS code, due to imprecision of identifying alias relationships as well as high costs of typestate tracking and path-feasibility validation. In this paper, we present PATA, a novel path-sensitive and aliasaware typestate analysis framework to detect OS bugs. To improve the precision of identifying alias relationships in OS code, PATA performs a path-based alias analysis based on control-flow paths and access paths. With these alias relationships, PATA reduces the costs of typestate tracking and path-feasibility validation, to boost the efficiency of path-sensitive typestate analysis for bug detection. We have evaluated PATA on the Linux kernel and three popular IoT OSes (Zephyr, RIOT and TencentOS-tiny) to detect three common types of bugs (null-pointer dereferences, uninitialized variable accesses and memory leaks). PATA finds 574 real bugs with a false positive rate of 28%. 206 of these bugs have been confirmed by the developers of the four OSes.We also compare PATA to seven state-of-the-art static approaches (Cppcheck, Coccinelle, Smatch,CSA, Infer, Saber and SVF). PATA finds many real bugs missed by them, with a lower false positive rate. @InProceedings{ASPLOS22p859, author = {Tuo Li and Jia-Ju Bai and Yulei Sui and Shi-Min Hu}, title = {Path-Sensitive and Alias-Aware Typestate Analysis for Detecting OS Bugs}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {859--872}, doi = {10.1145/3503222.3507770}, year = {2022}, } Publisher's Version
	Li, Yiming	ASPLOS '22: "INFless: A Native Serverless ..." INFless: A Native Serverless System for Low-Latency, High-Throughput Inference Yanan Yang, Laiping Zhao, Yiming Li, Huanyu Zhang, Jie Li, Mingyang Zhao, Xingzhen Chen, and Keqiu Li (Tianjin University, China; 58.com, China) Modern websites increasingly rely on machine learning (ML) to improve their business efficiency. Developing and maintaining ML services incurs high costs for developers. Although serverless systems are a promising solution to reduce costs, we find that the current general purpose serverless systems cannot meet the low latency, high throughput demands of ML services. While simply ”patching” general serverless systems does not resolve the problem completely, we propose that such a system should natively combine the features of inference with a serverless paradigm. We present INFless, the first ML domain-specific serverless platform. It provides a unified, heterogeneous resource abstraction between CPU and accelerators, and achieves high throughput using built-in batching and non-uniform scaling mechanisms. It also supports low latency through coordinated management of batch queuing time, execution time and coldstart rate. We evaluate INFless using both a local cluster testbed and a large-scale simulation. Experimental results show that INFless outperforms state-of-the-art systems by 2×-5× on system throughput, meeting the latency goals of ML services. @InProceedings{ASPLOS22p768, author = {Yanan Yang and Laiping Zhao and Yiming Li and Huanyu Zhang and Jie Li and Mingyang Zhao and Xingzhen Chen and Keqiu Li}, title = {INFless: A Native Serverless System for Low-Latency, High-Throughput Inference}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {768--781}, doi = {10.1145/3503222.3507709}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available
	Li, Zhenxin	ASPLOS '22: "NVAlloc: Rethinking Heap Metadata ..." NVAlloc: Rethinking Heap Metadata Management in Persistent Memory Allocators Zheng Dang, Shuibing He, Peiyi Hong, Zhenxin Li, Xuechen Zhang, Xian-He Sun, and Gang Chen (Zhejiang University, China; Washington State University, USA; Illinois Institute of Technology, USA) Persistent memory allocation is a fundamental building block for developing high-performance and in-memory applications. Existing persistent memory allocators suffer from suboptimal heap organizations that introduce repeated cache line flushes and small random accesses in persistent memory. Worse, many allocators use static slab segregation resulting in a dramatic increase in memory consumption when allocation request size is changed. In this paper, we design a novel allocator, named NVAlloc, to solve the above issues simultaneously. First, NVAlloc eliminates cache line reflushes by mapping contiguous data blocks in slabs to interleaved metadata entries stored in different cache lines. Second, it writes small metadata units to a persistent bookkeeping log in a sequential pattern to remove random heap metadata accesses in persistent memory. Third, instead of using static slab segregation, it supports slab morphing, which allows slabs to be transformed between size classes to significantly improve slab usage. NVAlloc is complementary to the existing consistency models. Results on 6 benchmarks demonstrate that NVAlloc improves the performance of state-of-the-art persistent memory allocators by up to 6.4x and 57x for small and large allocations, respectively. Using NVAlloc reduces memory usage by up to 57.8%. Besides, we integrate NVAlloc in a persistent FPTree. Compared to the state-of-the-art allocators, NVAlloc improves the performance of this application by up to 3.1x. @InProceedings{ASPLOS22p115, author = {Zheng Dang and Shuibing He and Peiyi Hong and Zhenxin Li and Xuechen Zhang and Xian-He Sun and Gang Chen}, title = {NVAlloc: Rethinking Heap Metadata Management in Persistent Memory Allocators}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {115--127}, doi = {10.1145/3503222.3507743}, year = {2022}, } Publisher's Version
	Li, Zijun	ASPLOS '22: "FaaSFlow: Enable Efficient ..." FaaSFlow: Enable Efficient Workflow Execution for Function-as-a-Service Zijun Li, Yushi Liu, Linsong Guo, Quan Chen, Jiagan Cheng, Wenli Zheng, and Minyi Guo (Shanghai Jiao Tong University, China) Serverless computing (Function-as-a-Service) provides fine-grain resource sharing by running functions (or Lambdas) in containers. Data-dependent functions are required to be invoked following a pre-defined logic, which is known as serverless workflows. However, our investigation shows that the traditional master-worker based workflow execution architecture performs poorly in serverless context. One significant overhead results from the master-side workflow schedule pattern, with which the functions are triggered in the master node and assigned to worker nodes for execution. Besides, the data movement between workers also reduces the throughput. To this end, we present a worker-side workflow schedule pattern for serverless workflow execution. Following the design, we implement FaaSFlow to enable efficient workflow execution in the serverless context. Besides, we propose an adaptive storage library FaaStore that enables fast data transfer between functions on the same node without through the database. Experiment results show that FaaSFlow effectively mitigates the workflow scheduling overhead by 74.6% on average and data transmission overhead by 95% at most. When the network bandwidth fluctuates, FaaSFlow-FaaStore reduces the throughput degradation by 23.0%, and is able to multiply the utilization of network bandwidth by 1.5X-4X. @InProceedings{ASPLOS22p782, author = {Zijun Li and Yushi Liu and Linsong Guo and Quan Chen and Jiagan Cheng and Wenli Zheng and Minyi Guo}, title = {FaaSFlow: Enable Efficient Workflow Execution for Function-as-a-Service}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {782--796}, doi = {10.1145/3503222.3507717}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Liang, Yu	ASPLOS '22: "Who Goes First? Detecting ..." Who Goes First? Detecting Go Concurrency Bugs via Message Reordering Ziheng Liu, Shihao Xia, Yu Liang, Linhai Song, and Hong Hu (Pennsylvania State University, USA) Go is a young programming language invented to build safe and efficient concurrent programs. It provides goroutines as lightweight threads and channels for inter-goroutine communication. Programmers are encouraged to explicitly pass messages through channels to connect goroutines, with the purpose of reducing the chance of making programming mistakes and introducing concurrency bugs. Go is one of the most beloved programming languages and has already been used to build many critical infrastructure software systems in the data-center environment. However, a recent study shows that channel-related concurrency bugs are still common in Go programs, severely hurting the reliability of the programs. This paper presents GFuzz, a dynamic detector that can effectively pinpoint channel-related concurrency bugs by mutating the processing orders of concurrent messages. We build GFuzz in three steps. We first adopt an effective approach to identify concurrent messages and transform a program to process those messages in any given order. We then take a fuzzing approach to generate new processing orders by mutating exercised ones and rely on execution feedback to prioritize orders close to triggering bugs. Finally, we design a runtime sanitizer to capture triggered bugs that are missed by the Go runtime. We evaluate GFuzz on seven popular Go software systems, including Docker, Kubernetes, and gRPC. GFuzz finds 184 previously unknown bugs and reports a negligible number of false positives. Programmers have already confirmed 124 reports as real bugs and fixed 67 of them based on our reporting. A careful inspection of the detected concurrency bugs from gRPC shows the effectiveness of each component of GFuzz and confirms the components' rationality. @InProceedings{ASPLOS22p888, author = {Ziheng Liu and Shihao Xia and Yu Liang and Linhai Song and Hong Hu}, title = {Who Goes First? Detecting Go Concurrency Bugs via Message Reordering}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {888--902}, doi = {10.1145/3503222.3507753}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional
	Licciardello, Melissa	ASPLOS '22: "Enzian: An Open, General, ..." Enzian: An Open, General, CPU/FPGA Platform for Systems Software Research David Cock, Abishek Ramdas, Daniel Schwyn, Michael Giardino, Adam Turowski, Zhenhao He, Nora Hossle, Dario Korolija, Melissa Licciardello, Kristina Martsenko, Reto Achermann, Gustavo Alonso, and Timothy Roscoe (ETH Zurich, Switzerland; University of British Columbia, Canada) Hybrid computing platforms, comprising CPU cores and FPGA logic, are increasingly used for accelerating data-intensive workloads in cloud deployments, and are a growing topic of interest in systems research. However, from a research perspective, existing hardware platforms are limited: they are often optimized for concrete, narrow use-cases and, therefore lack the flexibility needed to explore other applications and configurations. We show that a research group can design and build a more general, open, and affordable hardware platform for hybrid systems research. The platform, Enzian, is capable of duplicating the functionality of existing CPU/FPGA systems with comparable performance but in an open, flexible system. It couples a large FPGA with a server-class CPU in an asymmetric cache-coherent NUMA system. Enzian also enables research not possible with existing hybrid platforms, through explicit access to coherence messages, extensive thermal and power instrumentation, and an open, programmable baseboard management processor. Enzian is already being used in multiple projects, is open source (both hardware and software), and available for remote use. We present the design principles of Enzian, the challenges in building it, and evaluate it with a range of existing research use-cases alongside other, more specialized platforms, as well as demonstrating research not possible on existing platforms. @InProceedings{ASPLOS22p434, author = {David Cock and Abishek Ramdas and Daniel Schwyn and Michael Giardino and Adam Turowski and Zhenhao He and Nora Hossle and Dario Korolija and Melissa Licciardello and Kristina Martsenko and Reto Achermann and Gustavo Alonso and Timothy Roscoe}, title = {Enzian: An Open, General, CPU/FPGA Platform for Systems Software Research}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {434--451}, doi = {10.1145/3503222.3507742}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Lin, Felix Xiaozhu	ASPLOS '22: "GPUReplay: A 50-KB GPU Stack ..." GPUReplay: A 50-KB GPU Stack for Client ML Heejin Park and Felix Xiaozhu Lin (Purdue University, USA; University of Virginia, USA) GPUReplay (GR) is a novel way for deploying GPU-accelerated computation on mobile and embedded devices. It addresses high complexity of a modern GPU stack for deployment ease and security. The idea is to record GPU executions on the full GPU stack ahead of time and replay the executions on new input at run time. We address key challenges towards making GR feasible, sound, and practical to use. The resultant replayer is a drop-in replacement of the original GPU stack. It is tiny (50 KB of executable), robust (replaying long executions without divergence), portable (running in a commodity OS, in TEE, and baremetal), and quick to launch (speeding up startup by up to two orders of magnitude). We show that GPUReplay works with a variety of integrated GPU hardware, GPU APIs, ML frameworks, and 33 neural network (NN) implementations for inference or training. The code is available at https://github.com/bakhi/GPUReplay. @InProceedings{ASPLOS22p157, author = {Heejin Park and Felix Xiaozhu Lin}, title = {GPUReplay: A 50-KB GPU Stack for Client ML}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {157--170}, doi = {10.1145/3503222.3507754}, year = {2022}, } Publisher's Version
	Lin, Wei	ASPLOS '22: "AStitch: Enabling a New Multi-dimensional ..." AStitch: Enabling a New Multi-dimensional Optimization Space for Memory-Intensive ML Training and Inference on Modern SIMT Architectures Zhen Zheng, Xuanda Yang, Pengzhan Zhao, Guoping Long, Kai Zhu, Feiwen Zhu, Wenyi Zhao, Xiaoyong Liu, Jun Yang, Jidong Zhai, Shuaiwen Leon Song, and Wei Lin (Alibaba Group, China; Tsinghua University, China; University of Sydney, Australia) This work reveals that memory-intensive computation is a rising performance-critical factor in recent machine learning models. Due to a unique set of new challenges, existing ML optimizing compilers cannot perform efficient fusion under complex two-level dependencies combined with just-in-time demand. They face the dilemma of either performing costly fusion due to heavy redundant computation, or skipping fusion which results in massive number of kernels. Furthermore, they often suffer from low parallelism due to the lack of support for real-world production workloads with irregular tensor shapes. To address these rising challenges, we propose AStitch, a machine learning optimizing compiler that opens a new multi-dimensional optimization space for memory-intensive ML computations. It systematically abstracts four operator-stitching schemes while considering multi-dimensional optimization objectives, tackles complex computation graph dependencies with novel hierarchical data reuse, and efficiently processes various tensor shapes via adaptive thread mapping. Finally, AStitch provides just-in-time support incorporating our proposed optimizations for both ML training and inference. Although AStitch serves as a stand-alone compiler engine that is portable to any version of TensorFlow, its basic ideas can be generally applied to other ML frameworks and optimization compilers. Experimental results show that AStitch can achieve an average of 1.84x speedup (up to 2.73x) over the state-of-the-art Google's XLA solution across five production workloads. We also deploy AStitch onto a production cluster for ML workloads with thousands of GPUs. The system has been in operation for more than 10 months and saves about 20,000 GPU hours for 70,000 tasks per week. @InProceedings{ASPLOS22p359, author = {Zhen Zheng and Xuanda Yang and Pengzhan Zhao and Guoping Long and Kai Zhu and Feiwen Zhu and Wenyi Zhao and Xiaoyong Liu and Jun Yang and Jidong Zhai and Shuaiwen Leon Song and Wei Lin}, title = {AStitch: Enabling a New Multi-dimensional Optimization Space for Memory-Intensive ML Training and Inference on Modern SIMT Architectures}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {359--373}, doi = {10.1145/3503222.3507723}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Liss, Liran	ASPLOS '22: "FlexDriver: A Network Driver ..." FlexDriver: A Network Driver for Your Accelerator Haggai Eran, Maxim Fudim, Gabi Malka, Gal Shalom, Noam Cohen, Amit Hermony, Dotan Levi, Liran Liss, and Mark Silberstein (NVIDIA, Israel; Technion, Israel) We propose a new system design for connecting hardware and FPGA accelerators to the network, allowing the accelerator to directly control commodity Network Interface Cards (NICs) without using the CPU. This enables us to solve the key challenge of leveraging existing NIC hardware offloads such as virtualization, tunneling, and RDMA for accelerator networking. Our approach supports a diverse set of use cases, from direct network access for disaggregated accelerators to inline-acceleration of the network stack, all without the complex networking logic in the accelerator. To demonstrate the feasibility of this approach, we build FlexDriver (FLD), an on-accelerator hardware module that implements a NIC data-plane driver. Our main technical contribution is a mechanism that compresses the NIC control structures by two orders of magnitude, allowing FLD to achieve high networking scalability with low die area cost and no bandwidth interference with the accelerator logic. The prototype for NVIDIA Innova-2 FPGA SmartNICs showcases our design’s utility for three different accelerators: a disaggregated LTE cipher, an IP-defragmentation inline accelerator, and an IoT cryptographic-token authentication offload. These accelerators reach 25 Gbps line rate and leverage the NIC for RDMA processing, VXLAN tunneling, and traffic shaping without CPU involvement. @InProceedings{ASPLOS22p1115, author = {Haggai Eran and Maxim Fudim and Gabi Malka and Gal Shalom and Noam Cohen and Amit Hermony and Dotan Levi and Liran Liss and Mark Silberstein}, title = {FlexDriver: A Network Driver for Your Accelerator}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1115--1129}, doi = {10.1145/3503222.3507776}, year = {2022}, } Publisher's Version ASPLOS '22: "The Benefits of General-Purpose ..." The Benefits of General-Purpose On-NIC Memory Boris Pismenny, Liran Liss, Adam Morrison, and Dan Tsafrir (Technion, Israel; NVIDIA, Israel; Tel Aviv University, Israel; VMware Research, Israel) We propose to use the small, newly available on-NIC memory ("nicmem") to keep pace with the rapidly increasing performance of NICs. We motivate our proposal by accelerating two types of workload classes: NFV and key-value stores. As NFV workloads frequently operate on headers---rather than data---of incoming packets, we introduce a new packet-processing architecture that splits between the two, keeping the data on nicmem when possible and thus reducing PCIe traffic, memory bandwidth, and CPU processing time. Our approach consequently shortens NFV latency by up to 23% and increases its throughput by up to 19%. Similarly, because key-value stores commonly exhibit skewed distributions, we introduce a new network stack mechanism that lets applications keep frequently accessed items on nicmem. Our design shortens memcached latency by up to 43% and increases its throughput by up to 80%. @InProceedings{ASPLOS22p1130, author = {Boris Pismenny and Liran Liss and Adam Morrison and Dan Tsafrir}, title = {The Benefits of General-Purpose On-NIC Memory}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1130--1147}, doi = {10.1145/3503222.3507711}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Litz, Heiner	ASPLOS '22: "CRISP: Critical Slice Prefetching ..." CRISP: Critical Slice Prefetching Heiner Litz, Grant Ayers, and Parthasarathy Ranganathan (University of California at Santa Cruz, USA; Google, USA) The high access latency of DRAM continues to be a performance challenge for contemporary microprocessor systems. Prefetching is a well-established technique to address this problem, however, existing implemented designs fail to provide any performance benefits in the presence of irregular memory access patterns. The hardware complexity of prior techniques that can predict irregular memory accesses such as runahead execution has proven untenable for implementation in real hardware. We propose a lightweight mechanism to hide the high latency of irregular memory access patterns by leveraging criticality-based scheduling. In particular, our technique executes delinquent loads and their load slices as early as possible, hiding a significant fraction of their latency. Furthermore, we observe that the latency induced by branch mispredictions and other high latency instructions can be hidden with a similar approach. Our proposal only requires minimal hardware modifications by performing memory access classification, load and branch slice extraction, as well as priority analysis exclusively in software. As a result, our technique is feasible to implement, introducing only a simple new instruction prefix while requiring minimal modifications of the instruction scheduler. Our technique increases the IPC of memory-latency-bound applications by up to 38% and by 8.4% on average. @InProceedings{ASPLOS22p300, author = {Heiner Litz and Grant Ayers and Parthasarathy Ranganathan}, title = {CRISP: Critical Slice Prefetching}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {300--313}, doi = {10.1145/3503222.3507745}, year = {2022}, } Publisher's Version
	Liu, Fangming	ASPLOS '22: "CoolEdge: Hotspot-Relievable ..." CoolEdge: Hotspot-Relievable Warm Water Cooling for Energy-Efficient Edge Datacenters Qiangyu Pei, Shutong Chen, Qixia Zhang, Xinhui Zhu, Fangming Liu, Ziyang Jia, Yishuo Wang, and Yongjie Yuan (Huazhong University of Science and Technology, China) As the computing frontier drifts to the edge, edge datacenters play a crucial role in supporting various real-time applications. Different from cloud datacenters, the requirements of proximity to end-users, high density, and heterogeneity, present new challenges to cool the edge datacenters efficiently. Although warm water cooling has become a promising cooling technique for this infrastructure, the one-size-fits-all cooling control would lower the cooling efficiency considerably because of the severe thermal imbalance across servers, hardware, and even inside one hardware component in an edge datacenter. In this work, we propose CoolEdge, a hotspot-relievable warm water cooling system for improving the cooling efficiency and saving costs of edge datacenters. Specifically, through the elaborate design of water circulations, CoolEdge can dynamically adjust the water temperature and flow rate for each heterogeneous hardware component to eliminate the hardware-level hotspots. By redesigning cold plates, CoolEdge can quickly disperse the chip-level hotspots without manual intervention. We further quantify the power saving achieved by the warm water cooling theoretically, and propose a custom-designed cooling solution to decide an appropriate water temperature and flow rate periodically. Based on a hardware prototype and real-world traces from SURFsara, the evaluation results show that CoolEdge reduces the cooling energy by 81.81% and 71.92%, respectively, compared with conventional and state-of-the-art water cooling systems. @InProceedings{ASPLOS22p814, author = {Qiangyu Pei and Shutong Chen and Qixia Zhang and Xinhui Zhu and Fangming Liu and Ziyang Jia and Yishuo Wang and Yongjie Yuan}, title = {CoolEdge: Hotspot-Relievable Warm Water Cooling for Energy-Efficient Edge Datacenters}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {814--829}, doi = {10.1145/3503222.3507713}, year = {2022}, } Publisher's Version
	Liu, Guodong	ASPLOS '22: "Breaking the Computation and ..." Breaking the Computation and Communication Abstraction Barrier in Distributed Machine Learning Workloads Abhinav Jangda, Jun Huang, Guodong Liu, Amir Hossein Nodehi Sabet, Saeed Maleki, Youshan Miao, Madanlal Musuvathi, Todd Mytkowicz, and Olli Saarikivi (University of Massachusetts at Amherst, USA; Ohio State University, USA; Chinese Academy of Sciences, China; University of California at Riverside, USA; Microsoft Research, USA; Microsoft Research, China) Recent trends towards large machine learning models require both training and inference tasks to be distributed. Considering the huge cost of training these models, it is imperative to unlock optimizations in computation and communication to obtain best performance. However, the current logical separation between computation and communication kernels in machine learning frameworks misses optimization opportunities across this barrier. Breaking this abstraction can provide many optimizations to improve the performance of distributed workloads. However, manually applying these optimizations requires modifying the underlying computation and communication libraries for each scenario, which is both time consuming and error-prone. Therefore, we present CoCoNet, which contains (i) a domain specific language to express a distributed machine learning program in the form of computation and communication operations, (ii) a set of semantics preserving transformations to optimize the program, and (iii) a compiler to generate jointly optimized communication and computation GPU kernels. Providing both computation and communication as first class constructs allows users to work on a high-level abstraction and apply powerful optimizations, such as fusion or overlapping of communication and computation. CoCoNet enabled us to optimize data-, model- and pipeline-parallel workloads in large language models with only a few lines of code. Our experiments show that CoCoNet significantly outperforms state-of-the-art distributed machine learning implementations. @InProceedings{ASPLOS22p402, author = {Abhinav Jangda and Jun Huang and Guodong Liu and Amir Hossein Nodehi Sabet and Saeed Maleki and Youshan Miao and Madanlal Musuvathi and Todd Mytkowicz and Olli Saarikivi}, title = {Breaking the Computation and Communication Abstraction Barrier in Distributed Machine Learning Workloads}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {402--416}, doi = {10.1145/3503222.3507778}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional
	Liu, Junrui	ASPLOS '22: "Tree Traversal Synthesis Using ..." Tree Traversal Synthesis Using Domain-Specific Symbolic Compilation Yanju Chen, Junrui Liu, Yu Feng, and Rastislav Bodik (University of California at Santa Barbara, USA; University of Washington, USA) Efficient computation on tree data structures is important in compilers, numeric computations, and web browser layout engines. Efficiency is achieved by statically scheduling the computation into a small number of tree traversals and by performing the traversals in parallel when possible. Manual design of such traversals leads to bugs, as observed in web browsers. Automatic schedulers avoid these bugs but they currently cannot explore a space of legal traversals, which prevents exploring the trade-offs between parallelism and minimizing the number of traversals. We describe Hecate, a synthesizer of tree traversals that can produce both serial and parallel traversals. A key feature is that the synthesizer is extensible by the programmer who can define a template for new kinds of traversals. Hecate is constructed as a solver-aided domain-specific language, meaning that the synthesizer is generated automatically by translating the tree traversal DSL to an SMT solver that synthesizes the traversals. We improve on the general-purpose solver-aided architecture with a scheduling-specific symbolic evaluation that maintains the engineering advantages solver-aided design but generates efficient ILP encoding that is much more efficient to solve than SMT constraints. On the set of Grafter problems, Hecate synthesizes traversals that trade off traversal fusion to exploit parallelism. Additionally, Hecate allows defining a tree data structure with an arbitrary number of children. Together, parallelism and data structure improvements accelerate the computation 2× on a tree rendering problem. Finally, Hecate’s domain-specific symbolic compilation accelerates synthesis 3× compared to the general-purpose compilation to an SMT solver; when scheduling a CSS engine traversal, this ILP-based synthesis executes orders of magnitude faster. @InProceedings{ASPLOS22p1030, author = {Yanju Chen and Junrui Liu and Yu Feng and Rastislav Bodik}, title = {Tree Traversal Synthesis Using Domain-Specific Symbolic Compilation}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1030--1042}, doi = {10.1145/3503222.3507751}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional
	Liu, Liu	ASPLOS '22: "DOTA: Detect and Omit Weak ..." DOTA: Detect and Omit Weak Attentions for Scalable Transformer Acceleration Zheng Qu, Liu Liu, Fengbin Tu, Zhaodong Chen, Yufei Ding, and Yuan Xie (University of California at Santa Barbara, USA) Transformer Neural Networks have demonstrated leading performance in many applications spanning over language understanding, image processing, and generative modeling. Despite the impressive performance, long-sequence Transformer processing is expensive due to quadratic computation complexity and memory consumption of self-attention. In this paper, we present DOTA, an algorithm-architecture co-design that effectively addresses the challenges of scalable Transformer inference. Based on the insight that not all connections in an attention graph are equally important, we propose to jointly optimize a lightweight Detector with the Transformer model to accurately detect and omit weak connections during runtime. Furthermore, we design a specialized system architecture for end-to-end Transformer acceleration using the proposed attention detection mechanism. Experiments on a wide range of benchmarks demonstrate the superior performance of DOTA over other solutions. In summary, DOTA achieves 152.6x and 4.5x performance speedup and orders of magnitude energy-efficiency improvements over GPU and customized hardware, respectively. @InProceedings{ASPLOS22p14, author = {Zheng Qu and Liu Liu and Fengbin Tu and Zhaodong Chen and Yufei Ding and Yuan Xie}, title = {DOTA: Detect and Omit Weak Attentions for Scalable Transformer Acceleration}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {14--26}, doi = {10.1145/3503222.3507738}, year = {2022}, } Publisher's Version ASPLOS '22: "A One-for-All and O(V ..." A One-for-All and O(V log(V ))-Cost Solution for Parallel Merge Style Operations on Sorted Key-Value Arrays Bangyan Wang, Lei Deng, Fei Sun, Guohao Dai, Liu Liu, Yu Wang, and Yuan Xie (University of California at Santa Barbara, USA; Tsinghua University, China; Alibaba DAMO Academy, China) The processing of sorted key-value arrays using a “merge style operation (MSO)” is a very basic and important problem in domains like scientific computing, deep learning, database, graph analysis, sorting, set-operation etc. MSOs dominate the execution time in some important applications like SpGEMM and graph mining. For example, sparse vector addition as an MSO takes up to 98% execution time in SpGEMM in our experiment. For this reason, accelerating MSOs on CPU, GPU, and accelerators using parallel execution has been extensively studied but the solutions in prior work have three major limitations. (1) They treat different MSOs as isolated problems using incompatible methods and an unified solution is still lacking. (2) They do not have the flexibility to support variable key/value sizes and value calculations in the runtime given a fixed hardware design. (3) They require a quadratic hardware cost (O(V2)) for given parallelism V in most cases. To address above three limitations, we make the following efforts. (1) We present a one-for-all solution to support all interested MSOs based on a unified abstraction model “restricted zip machine (RZM)”. (2) We propose a set of composable and parallel primitives for RZM to provide the flexibility to support variable key/value sizes and value calculations. (3) We provide the hardware design to implement the proposed primitives using only O(Vlog(V)) resource. With the above techniques, a flexible and efficient solution for MSOs has been built. Our design can be used either as a drop-in replacement of the merge unit in prior accelerators to reduce the cost from O(V2) to O(Vlog(V)), or as an extension to the SIMD ISA of CPU and GPU. In our evaluation on CPU, when V=16 (512-bit SIMD, 32-bit element), we achieve significant speedup on a range of representative kernels including set operations (8.4×), database joins (7.3×), sparse vector/matrix/tensor addition/multiplication on real/complex numbers (6.5×), merge sort (8.0× over scalar, 3.4× over the state-of-the-art SIMD), and SpGEMM (4.4× over the best one in the baseline collection). @InProceedings{ASPLOS22p669, author = {Bangyan Wang and Lei Deng and Fei Sun and Guohao Dai and Liu Liu and Yu Wang and Yuan Xie}, title = {A One-for-All and <i>O</i>(<i>V</i> log(<i>V</i> ))-Cost Solution for Parallel Merge Style Operations on Sorted Key-Value Arrays}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {669--682}, doi = {10.1145/3503222.3507728}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Liu, Peng	ASPLOS '22: "RSSD: Defend against Ransomware ..." RSSD: Defend against Ransomware with Hardware-Isolated Network-Storage Codesign and Post-Attack Analysis Benjamin Reidys, Peng Liu, and Jian Huang (University of Illinois at Urbana-Champaign, USA; Pennsylvania State University, USA) Encryption ransomware has become a notorious malware. It encrypts user data on storage devices like solid-state drives (SSDs) and demands a ransom to restore data for users. To bypass existing defenses, ransomware would keep evolving and performing new attack models. For instance, we identify and validate three new attacks, including (1) garbage-collection (GC) attack that exploits storage capacity and keeps writing data to trigger GC and force SSDs to release the retained data; (2) timing attack that intentionally slows down the pace of encrypting data and hides its I/O patterns to escape existing defense; (3) trimming attack that utilizes the trim command available in SSDs to physically erase data. To enhance the robustness of SSDs against these attacks, we propose RSSD, a ransomware-aware SSD. It redesigns the flash management of SSDs for enabling the hardware-assisted logging, which can conservatively retain older versions of user data and received storage operations in time order with low overhead. It also employs hardware-isolated NVMe over Ethernet to expand local storage capacity by transparently offloading the logs to remote cloud/servers in a secure manner. RSSD enables post-attack analysis by building a trusted evidence chain of storage operations to assist the investigation of ransomware attacks. We develop RSSD with a real-world SSD FPGA board. Our evaluation shows that RSSD can defend against new and future ransomware attacks, while introducing negligible performance overhead. @InProceedings{ASPLOS22p726, author = {Benjamin Reidys and Peng Liu and Jian Huang}, title = {RSSD: Defend against Ransomware with Hardware-Isolated Network-Storage Codesign and Post-Attack Analysis}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {726--739}, doi = {10.1145/3503222.3507773}, year = {2022}, } Publisher's Version
	Liu, Qingyuan	ASPLOS '22: "Serverless Computing on Heterogeneous ..." Serverless Computing on Heterogeneous Computers Dong Du, Qingyuan Liu, Xueqiang Jiang, Yubin Xia, Binyu Zang, and Haibo Chen (Shanghai Jiao Tong University, China; Shanghai Artificial Intelligence Laboratory, China) Existing serverless computing platforms are built upon homogeneous computers, limiting the function density and restricting serverless computing to limited scenarios. We introduce Molecule, the first serverless computing system utilizing heterogeneous computers. Molecule enables both general-purpose devices (e.g., Nvidia DPU) and domain-specific accelerators (e.g., FPGA and GPU) for serverless applications that significantly improve function density (50% higher) and application performance (up to 34.6x). To achieve these results, we first propose XPU-Shim, a distributed shim to bridge the gap between underlying multi-OS systems (when using general-purpose devices) and our serverless runtime (i.e., Molecule). We further introduce vectorized sandbox, a sandbox abstraction to abstract hardware heterogeneity (when using domain-specific accelerators). Moreover, we also review state-of-the-art serverless optimizations on startup and communication latency and overcome the challenges to implement them on heterogeneous computers. We have implemented Molecule on real platforms with Nvidia DPUs and Xilinx FPGAs and evaluate it using benchmarks and real-world applications. @InProceedings{ASPLOS22p797, author = {Dong Du and Qingyuan Liu and Xueqiang Jiang and Yubin Xia and Binyu Zang and Haibo Chen}, title = {Serverless Computing on Heterogeneous Computers}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {797--813}, doi = {10.1145/3503222.3507732}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Liu, Song	ASPLOS '22: "IOCost: Block IO Control for ..." IOCost: Block IO Control for Containers in Datacenters Tejun Heo, Dan Schatzberg, Andrew Newell, Song Liu, Saravanan Dhakshinamurthy, Iyswarya Narayanan, Josef Bacik, Chris Mason, Chunqiang Tang, and Dimitrios Skarlatos (Meta, USA; Carnegie Mellon University, USA) Resource isolation is a fundamental requirement in datacenter environments. However, our production experience in Meta’s large-scale datacenters shows that existing IO control mechanisms for block storage are inadequate in containerized environments. IO control needs to provide proportional resources to containers while taking into account the hardware heterogeneity of storage devices and the idiosyncrasies of the workloads deployed in datacenters. The speed of modern SSDs requires IO control to execute with low-overheads. Furthermore, IO control should strive for work conservation, take into account the interactions with the memory management subsystem, and avoid priority inversions that lead to isolation failures. To address these challenges, this paper presents IOCost, an IO control solution that is designed for containerized environments and provides scalable, work-conserving, and low-overhead IO control for heterogeneous storage devices and diverse workloads in datacenters. IOCost performs offline profiling to build a device model and uses it to estimate device occupancy of each IO request. To minimize runtime overhead, it separates IO control into a fast per-IO issue path and a slower periodic planning path. A novel work-conserving budget donation algorithm enables containers to dynamically share unused budget. We have deployed IOCost across the entirety of Meta’s datacenters comprised of millions of ma- chines, upstreamed IOCost to the Linux kernel, and open-sourced our device-profiling tools. IOCost has been running in production for two years, providing IO control for Meta’s fleet. We describe the design of IOCost and share our experience deploying it at scale. @InProceedings{ASPLOS22p595, author = {Tejun Heo and Dan Schatzberg and Andrew Newell and Song Liu and Saravanan Dhakshinamurthy and Iyswarya Narayanan and Josef Bacik and Chris Mason and Chunqiang Tang and Dimitrios Skarlatos}, title = {IOCost: Block IO Control for Containers in Datacenters}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {595--608}, doi = {10.1145/3503222.3507727}, year = {2022}, } Publisher's Version
	Liu, Xiaoyong	ASPLOS '22: "AStitch: Enabling a New Multi-dimensional ..." AStitch: Enabling a New Multi-dimensional Optimization Space for Memory-Intensive ML Training and Inference on Modern SIMT Architectures Zhen Zheng, Xuanda Yang, Pengzhan Zhao, Guoping Long, Kai Zhu, Feiwen Zhu, Wenyi Zhao, Xiaoyong Liu, Jun Yang, Jidong Zhai, Shuaiwen Leon Song, and Wei Lin (Alibaba Group, China; Tsinghua University, China; University of Sydney, Australia) This work reveals that memory-intensive computation is a rising performance-critical factor in recent machine learning models. Due to a unique set of new challenges, existing ML optimizing compilers cannot perform efficient fusion under complex two-level dependencies combined with just-in-time demand. They face the dilemma of either performing costly fusion due to heavy redundant computation, or skipping fusion which results in massive number of kernels. Furthermore, they often suffer from low parallelism due to the lack of support for real-world production workloads with irregular tensor shapes. To address these rising challenges, we propose AStitch, a machine learning optimizing compiler that opens a new multi-dimensional optimization space for memory-intensive ML computations. It systematically abstracts four operator-stitching schemes while considering multi-dimensional optimization objectives, tackles complex computation graph dependencies with novel hierarchical data reuse, and efficiently processes various tensor shapes via adaptive thread mapping. Finally, AStitch provides just-in-time support incorporating our proposed optimizations for both ML training and inference. Although AStitch serves as a stand-alone compiler engine that is portable to any version of TensorFlow, its basic ideas can be generally applied to other ML frameworks and optimization compilers. Experimental results show that AStitch can achieve an average of 1.84x speedup (up to 2.73x) over the state-of-the-art Google's XLA solution across five production workloads. We also deploy AStitch onto a production cluster for ML workloads with thousands of GPUs. The system has been in operation for more than 10 months and saves about 20,000 GPU hours for 70,000 tasks per week. @InProceedings{ASPLOS22p359, author = {Zhen Zheng and Xuanda Yang and Pengzhan Zhao and Guoping Long and Kai Zhu and Feiwen Zhu and Wenyi Zhao and Xiaoyong Liu and Jun Yang and Jidong Zhai and Shuaiwen Leon Song and Wei Lin}, title = {AStitch: Enabling a New Multi-dimensional Optimization Space for Memory-Intensive ML Training and Inference on Modern SIMT Architectures}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {359--373}, doi = {10.1145/3503222.3507723}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Liu, Xu	ASPLOS '22: "ValueExpert: Exploring Value ..." ValueExpert: Exploring Value Patterns in GPU-Accelerated Applications Keren Zhou, Yueming Hao, John Mellor-Crummey, Xiaozhu Meng, and Xu Liu (Rice University, USA; North Carolina State University, USA) General-purpose GPUs have become common in modern computing systems to accelerate applications in many domains, including machine learning, high-performance computing, and autonomous driving. However, inefficiencies abound in GPU-accelerated applications, which prevent them from obtaining bare-metal performance. Performance tools play an important role in understanding performance inefficiencies in complex code bases. Many GPU performance tools pinpoint time-consuming code and provide high-level performance insights but overlook one important performance issue---value-related inefficiencies, which exist in many GPU code bases. In this paper, we present ValueExpert, a novel tool to pinpoint value-related inefficiencies in GPU applications. ValueExpert monitors application execution to capture values produced and used by each load and store operation in GPU kernels, recognizes multiple value patterns, and provides intuitive optimization guidance. We address systemic challenges in collecting, maintaining, and analyzing voluminous performance data from many GPU threads to make ValueExpert applicable to complex applications. We evaluate ValueExpert on a wide range of well-tuned benchmarks and applications, including PyTorch, Darknet, LAMMPS, Castro, and many others. ValueExpert is able to identify previously unknown performance issues and provide suggestions for nontrivial performance improvements with typically less than five lines of code changes. We verify our optimizations with application developers and upstream fixes to their repositories. @InProceedings{ASPLOS22p171, author = {Keren Zhou and Yueming Hao and John Mellor-Crummey and Xiaozhu Meng and Xu Liu}, title = {ValueExpert: Exploring Value Patterns in GPU-Accelerated Applications}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {171--185}, doi = {10.1145/3503222.3507708}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Liu, Yushi	ASPLOS '22: "FaaSFlow: Enable Efficient ..." FaaSFlow: Enable Efficient Workflow Execution for Function-as-a-Service Zijun Li, Yushi Liu, Linsong Guo, Quan Chen, Jiagan Cheng, Wenli Zheng, and Minyi Guo (Shanghai Jiao Tong University, China) Serverless computing (Function-as-a-Service) provides fine-grain resource sharing by running functions (or Lambdas) in containers. Data-dependent functions are required to be invoked following a pre-defined logic, which is known as serverless workflows. However, our investigation shows that the traditional master-worker based workflow execution architecture performs poorly in serverless context. One significant overhead results from the master-side workflow schedule pattern, with which the functions are triggered in the master node and assigned to worker nodes for execution. Besides, the data movement between workers also reduces the throughput. To this end, we present a worker-side workflow schedule pattern for serverless workflow execution. Following the design, we implement FaaSFlow to enable efficient workflow execution in the serverless context. Besides, we propose an adaptive storage library FaaStore that enables fast data transfer between functions on the same node without through the database. Experiment results show that FaaSFlow effectively mitigates the workflow scheduling overhead by 74.6% on average and data transmission overhead by 95% at most. When the network bandwidth fluctuates, FaaSFlow-FaaStore reduces the throughput degradation by 23.0%, and is able to multiply the utilization of network bandwidth by 1.5X-4X. @InProceedings{ASPLOS22p782, author = {Zijun Li and Yushi Liu and Linsong Guo and Quan Chen and Jiagan Cheng and Wenli Zheng and Minyi Guo}, title = {FaaSFlow: Enable Efficient Workflow Execution for Function-as-a-Service}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {782--796}, doi = {10.1145/3503222.3507717}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Liu, Zihan	ASPLOS '22: "VELTAIR: Towards High-Performance ..." VELTAIR: Towards High-Performance Multi-tenant Deep Learning Services via Adaptive Compilation and Scheduling Zihan Liu, Jingwen Leng, Zhihui Zhang, Quan Chen, Chao Li, and Minyi Guo (Shanghai Jiao Tong University, China; Shanghai Qi Zhi Institute, China) Deep learning (DL) models have achieved great success in many application domains. As such, many industrial companies such as Google and Facebook have acknowledged the importance of multi-tenant DL services. Although the multi-tenant service has been studied in conventional workloads, it is not been deeply studied on deep learning service, especially on general-purpose hardware. In this work, we systematically analyze the opportunities and challenges of providing multi-tenant deep learning services on the general-purpose CPU architecture from the aspects of scheduling granularity and code generation. We propose an adaptive granularity scheduling scheme to both guarantee resource usage efficiency and reduce the scheduling conflict rate. We also propose an adaptive compilation strategy, by which we can dynamically and intelligently pick a program with proper exclusive and shared resource usage to reduce overall interference-induced performance loss. Compared to the existing works, our design can serve more requests under the same QoS target in various scenarios (e.g., +71%, +62%, +45% for light, medium, and heavy workloads, respectively), and reduce the averaged query latency by 50%. @InProceedings{ASPLOS22p388, author = {Zihan Liu and Jingwen Leng and Zhihui Zhang and Quan Chen and Chao Li and Minyi Guo}, title = {VELTAIR: Towards High-Performance Multi-tenant Deep Learning Services via Adaptive Compilation and Scheduling}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {388--401}, doi = {10.1145/3503222.3507752}, year = {2022}, } Publisher's Version
	Liu, Ziheng	ASPLOS '22: "Who Goes First? Detecting ..." Who Goes First? Detecting Go Concurrency Bugs via Message Reordering Ziheng Liu, Shihao Xia, Yu Liang, Linhai Song, and Hong Hu (Pennsylvania State University, USA) Go is a young programming language invented to build safe and efficient concurrent programs. It provides goroutines as lightweight threads and channels for inter-goroutine communication. Programmers are encouraged to explicitly pass messages through channels to connect goroutines, with the purpose of reducing the chance of making programming mistakes and introducing concurrency bugs. Go is one of the most beloved programming languages and has already been used to build many critical infrastructure software systems in the data-center environment. However, a recent study shows that channel-related concurrency bugs are still common in Go programs, severely hurting the reliability of the programs. This paper presents GFuzz, a dynamic detector that can effectively pinpoint channel-related concurrency bugs by mutating the processing orders of concurrent messages. We build GFuzz in three steps. We first adopt an effective approach to identify concurrent messages and transform a program to process those messages in any given order. We then take a fuzzing approach to generate new processing orders by mutating exercised ones and rely on execution feedback to prioritize orders close to triggering bugs. Finally, we design a runtime sanitizer to capture triggered bugs that are missed by the Go runtime. We evaluate GFuzz on seven popular Go software systems, including Docker, Kubernetes, and gRPC. GFuzz finds 184 previously unknown bugs and reports a negligible number of false positives. Programmers have already confirmed 124 reports as real bugs and fixed 67 of them based on our reporting. A careful inspection of the detected concurrency bugs from gRPC shows the effectiveness of each component of GFuzz and confirms the components' rationality. @InProceedings{ASPLOS22p888, author = {Ziheng Liu and Shihao Xia and Yu Liang and Linhai Song and Hong Hu}, title = {Who Goes First? Detecting Go Concurrency Bugs via Message Reordering}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {888--902}, doi = {10.1145/3503222.3507753}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional
	Locharla, Aditya	ASPLOS '22: "LILLIPUT: A Lightweight Low-Latency ..." LILLIPUT: A Lightweight Low-Latency Lookup-Table Decoder for Near-Term Quantum Error Correction Poulami Das, Aditya Locharla, and Cody Jones (Georgia Institute of Technology, USA; Google, USA) The error rates of quantum devices are orders of magnitude higher than what is needed to run most quantum applications. To close this gap, Quantum Error Correction (QEC) encodes logical qubits and distributes information using several physical qubits. By periodically executing a syndrome extraction circuit on the logical qubits, information about errors (called syndrome) is extracted while running programs. A decoder uses these syndromes to identify and correct errors in real time, which is necessary to prevent accumulation of errors. Unfortunately, software decoders are slow and hardware decoders are fast but less accurate. Thus, almost all QEC studies so far have relied on offline decoding. To enable real-time decoding in near-term QEC, we propose LILLIPUT– a Lightweight Low Latency Look-Up Table decoder. LILLIPUT consists of two parts– First, it translates syndromes into error detection events that index into a Look-Up Table (LUT) whose entry provides the error information in real-time. Second, it programs the LUTs with error assignments for all possible error events by running a software decoder offline. LILLIPUT tolerates an error on any operation in the quantum hardware, including gates and measurements, and the number of tolerated errors grows with the size of the code. LILLIPUT utilizes less than 7% logic on off-the-shelf FPGAs enabling practical adoption, as FPGAs are already used to design the control and readout circuits in existing systems. LILLIPUT incurs a latency of a few nanoseconds and enables real-time decoding. We also propose Compressed LUTs (CLUTs) to reduce the memory required by LILLIPUT. By exploiting the fact that not all error events are equally likely and only storing data for the most probable error events, CLUTs reduce the memory needed by up-to 107x (from 148 MB to 1.38 MB) without degrading the accuracy. @InProceedings{ASPLOS22p541, author = {Poulami Das and Aditya Locharla and Cody Jones}, title = {LILLIPUT: A Lightweight Low-Latency Lookup-Table Decoder for Near-Term Quantum Error Correction}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {541--553}, doi = {10.1145/3503222.3507707}, year = {2022}, } Publisher's Version
	Long, Guoping	ASPLOS '22: "AStitch: Enabling a New Multi-dimensional ..." AStitch: Enabling a New Multi-dimensional Optimization Space for Memory-Intensive ML Training and Inference on Modern SIMT Architectures Zhen Zheng, Xuanda Yang, Pengzhan Zhao, Guoping Long, Kai Zhu, Feiwen Zhu, Wenyi Zhao, Xiaoyong Liu, Jun Yang, Jidong Zhai, Shuaiwen Leon Song, and Wei Lin (Alibaba Group, China; Tsinghua University, China; University of Sydney, Australia) This work reveals that memory-intensive computation is a rising performance-critical factor in recent machine learning models. Due to a unique set of new challenges, existing ML optimizing compilers cannot perform efficient fusion under complex two-level dependencies combined with just-in-time demand. They face the dilemma of either performing costly fusion due to heavy redundant computation, or skipping fusion which results in massive number of kernels. Furthermore, they often suffer from low parallelism due to the lack of support for real-world production workloads with irregular tensor shapes. To address these rising challenges, we propose AStitch, a machine learning optimizing compiler that opens a new multi-dimensional optimization space for memory-intensive ML computations. It systematically abstracts four operator-stitching schemes while considering multi-dimensional optimization objectives, tackles complex computation graph dependencies with novel hierarchical data reuse, and efficiently processes various tensor shapes via adaptive thread mapping. Finally, AStitch provides just-in-time support incorporating our proposed optimizations for both ML training and inference. Although AStitch serves as a stand-alone compiler engine that is portable to any version of TensorFlow, its basic ideas can be generally applied to other ML frameworks and optimization compilers. Experimental results show that AStitch can achieve an average of 1.84x speedup (up to 2.73x) over the state-of-the-art Google's XLA solution across five production workloads. We also deploy AStitch onto a production cluster for ML workloads with thousands of GPUs. The system has been in operation for more than 10 months and saves about 20,000 GPU hours for 70,000 tasks per week. @InProceedings{ASPLOS22p359, author = {Zhen Zheng and Xuanda Yang and Pengzhan Zhao and Guoping Long and Kai Zhu and Feiwen Zhu and Wenyi Zhao and Xiaoyong Liu and Jun Yang and Jidong Zhai and Shuaiwen Leon Song and Wei Lin}, title = {AStitch: Enabling a New Multi-dimensional Optimization Space for Memory-Intensive ML Training and Inference on Modern SIMT Architectures}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {359--373}, doi = {10.1145/3503222.3507723}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Loughlin, Kevin	ASPLOS '22: "Debugging in the Brave New ..." Debugging in the Brave New World of Reconfigurable Hardware Jiacheng Ma, Gefei Zuo, Kevin Loughlin, Haoyang Zhang, Andrew Quinn, and Baris Kasikci (University of Michigan, USA; University of California at Santa Cruz, USA) Software and hardware development cycles have traditionally been quite distinct. Software allows post-deployment patches, which leads to a rapid development cycle. In contrast, hardware bugs that are found after fabrication are extremely costly to fix (and sometimes even unfixable), so the traditional hardware development cycle involves massive investment in extensive simulation and formal verification. Reconfigurable hardware, such as a Field Programmable Gate Array (FPGA), promises to propel hardware development towards an agile software-like development approach, since it enables a hardware developer to patch bugs that are detected during on-chip testing or in production. Unfortunately, FPGA programmers lack bug localization tools amenable to this rapid development cycle, since past tools mainly find bugs via simulation and verification. To develop hardware bug localization tools for a rapid development cycle, a thorough understanding of the symptoms, root causes, and fixes of hardware bugs is needed. In this paper, we first study bugs in existing FPGA designs and produce a testbed of reliably-reproducible bugs. We classify the bugs according to their intrinsic properties, symptoms, and root causes. We demonstrate that many hardware bugs are comparable to software bug counterparts, and would benefit from similar techniques for bug diagnosis and repair. Based upon our findings, we build a novel collection of hybrid static/dynamic program analysis and monitoring tools for debugging FPGA designs, showing that our tools enable a software-like development cycle by effectively reducing developers' manual efforts for bug localization. @InProceedings{ASPLOS22p946, author = {Jiacheng Ma and Gefei Zuo and Kevin Loughlin and Haoyang Zhang and Andrew Quinn and Baris Kasikci}, title = {Debugging in the Brave New World of Reconfigurable Hardware}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {946--962}, doi = {10.1145/3503222.3507701}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Lucia, Brandon	ASPLOS '22: "Client-Optimized Algorithms ..." Client-Optimized Algorithms and Acceleration for Encrypted Compute Offloading McKenzie van der Hagen and Brandon Lucia (Carnegie Mellon University, USA) Homomorphic Encryption (HE) enables secure cloud offload processing on encrypted data. HE schemes are limited in the complexity and type of operations they can perform, motivating client-aided implementations that distribute computation between client (unencrypted) and server (encrypted). Prior client-aided systems optimize server performance, ignoring client costs: client-aided models put encryption and decryption on the critical path and require communicating large ciphertexts. We introduce Client-aided HE for Opaque Compute Offloading (CHOCO), a client-optimized system for encrypted offload processing. CHOCO reduces ciphertext size, reducing communication and computing costs through HE parameter minimization and through “rotational redundancy”, a new HE algorithm optimization. We present Client-aided HE for Opaque Compute Offloading Through Accelerated Cryptographic Operations (CHOCO-TACO), an accelerator for HE encryption and decryption, making client-aided HE feasible for even resource-constrained clients. CHOCO supports two popular HE schemes (BFV and CKKS) and several applications, including DNNs, PageRank, KNN, and K-Means. CHOCO reduces communication by up to 2948× over prior work. With CHOCO-TACO client enc-/decryption is up to 1094× faster and uses up to 648× less energy. @InProceedings{ASPLOS22p683, author = {McKenzie van der Hagen and Brandon Lucia}, title = {Client-Optimized Algorithms and Acceleration for Encrypted Compute Offloading}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {683--696}, doi = {10.1145/3503222.3507737}, year = {2022}, } Publisher's Version
	Luo, Xiapu	ASPLOS '22: "EXAMINER: Automatically Locating ..." EXAMINER: Automatically Locating Inconsistent Instructions between Real Devices and CPU Emulators for ARM Muhui Jiang, Tianyi Xu, Yajin Zhou, Yufeng Hu, Ming Zhong, Lei Wu, Xiapu Luo, and Kui Ren (Hong Kong Polytechnic University, China; Zhejiang University, China) Emulators are widely used to build dynamic analysis frameworks due to its fine-grained tracing capability, full system monitoring functionality, and scalability of running on different operating systems and architectures. However, whether emulators are consistent with real devices is unknown. To understand this problem, we aim to automatically locate inconsistent instructions, which behave differently between emulators and real devices. We target the ARM architecture, which provides machine-readable specifications. Based on the specification, we propose a sufficient test case generator by designing and implementing the first symbolic execution engine for the ARM architecture specification language (ASL). We generate 2,774,649 representative instruction streams and conduct differential testing between four ARM real devices in different architecture versions (i.e., ARMv5, ARMv6, ARMv7, and ARMv8) and three state-of-the-art emulators (i.e., QEMU, Unicorn, and Angr). We locate a huge number of inconsistent instruction streams (171,858 for QEMU, 223,264 for unicorn, and 120,169 for Angr). We find that undefined implementation in ARM manual and bugs of emulators are the major causes of inconsistencies. Furthermore, we discover 12 bugs, which influence commonly used instructions (e.g., BLX). With the inconsistent instructions, we build three security applications and demonstrate the capability of these instructions on detecting emulators, anti-emulation, and anti-fuzzing. @InProceedings{ASPLOS22p846, author = {Muhui Jiang and Tianyi Xu and Yajin Zhou and Yufeng Hu and Ming Zhong and Lei Wu and Xiapu Luo and Kui Ren}, title = {EXAMINER: Automatically Locating Inconsistent Instructions between Real Devices and CPU Emulators for ARM}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {846--858}, doi = {10.1145/3503222.3507736}, year = {2022}, } Publisher's Version
	Luo, Xuhao	ASPLOS '22: "Clio: A Hardware-Software ..." Clio: A Hardware-Software Co-Designed Disaggregated Memory System Zhiyuan Guo, Yizhou Shan, Xuhao Luo, Yutong Huang, and Yiying Zhang (University of California at San Diego, USA) Memory disaggregation has attracted great attention recently because of its benefits in efficient memory utilization and ease of management. So far, memory disaggregation research has all taken one of two approaches: building/emulating memory nodes using regular servers or building them using raw memory devices with no processing power. The former incurs higher monetary cost and faces tail latency and scalability limitations, while the latter introduces performance, security, and management problems. Server-based memory nodes and memory nodes with no processing power are two extreme approaches. We seek a sweet spot in the middle by proposing a hardware-based memory disaggregation solution that has the right amount of processing power at memory nodes. Furthermore, we take a clean-slate approach by starting from the requirements of memory disaggregation and designing a memory-disaggregation-native system. We built Clio, a disaggregated memory system that virtualizes, protects, and manages disaggregated memory at hardware-based memory nodes. The Clio hardware includes a new virtual memory system, a customized network system, and a framework for computation offloading. In building Clio, we not only co-design OS functionalities, hardware architecture, and the network system, but also co-design compute nodes and memory nodes. Our FPGA prototype of Clio demonstrates that each memory node can achieve 100 Gbps throughput and an end-to-end latency of 2.5 µ s at median and 3.2 µ s at the 99th percentile. Clio also scales much better and has orders of magnitude lower tail latency than RDMA. It has 1.1× to 3.4× energy saving compared to CPU-based and SmartNIC-based disaggregated memory systems and is 2.7× faster than software-based SmartNIC solutions. @InProceedings{ASPLOS22p417, author = {Zhiyuan Guo and Yizhou Shan and Xuhao Luo and Yutong Huang and Yiying Zhang}, title = {Clio: A Hardware-Software Co-Designed Disaggregated Memory System}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {417--433}, doi = {10.1145/3503222.3507762}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional
	Lyles, Darren	ASPLOS '22: "Temporal and SFQ Pulse-Streams ..." Temporal and SFQ Pulse-Streams Encoding for Area-Efficient Superconducting Accelerators Patricia Gonzalez-Guerrero, Meriam Gay Bautista, Darren Lyles, and George Michelogiannakis (Lawrence Berkeley National Laboratory, USA) Superconducting technology is a prime candidate for the future of computing. However, current superconducting prototypes are limited to small-scale examples due to stringent area constraints and complex architectures inspired from voltage-level encoding in CMOS; this is at odds with the ps-wide Single Quantum Flux (SFQ) pulses used in superconductors to carry information. In this work, we propose a wave-pipelined Unary SFQ (U-SFQ) architecture that leverages the advantages of two data representations: pulse-streams and Race Logic (RL). We introduce novel building blocks such as multipliers, adders, and memory cells, which leverage the natural properties of SFQ pulses to mitigate area constraints. We then design and simulate three popular hardware accelerators: i) a Processing Element (PE), typically used in spatial architectures; ii) A dot-product-unit (DPU), one of the most popular accelerators in artificial neural networks and digital signal processing (DSP); and iii) A Finite Impulse Response (FIR) filter, a popular and computationally demanding DSP accelerator. The proposed U-SFQ building blocks require up to 200× fewer JJs compared to their SFQ binary counterparts, exposing an area-delay trade-off. This work mitigates the stringent area constraints of superconducting technology. @InProceedings{ASPLOS22p963, author = {Patricia Gonzalez-Guerrero and Meriam Gay Bautista and Darren Lyles and George Michelogiannakis}, title = {Temporal and SFQ Pulse-Streams Encoding for Area-Efficient Superconducting Accelerators}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {963--976}, doi = {10.1145/3503222.3507765}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available
	Ma, Jiacheng	ASPLOS '22: "Debugging in the Brave New ..." Debugging in the Brave New World of Reconfigurable Hardware Jiacheng Ma, Gefei Zuo, Kevin Loughlin, Haoyang Zhang, Andrew Quinn, and Baris Kasikci (University of Michigan, USA; University of California at Santa Cruz, USA) Software and hardware development cycles have traditionally been quite distinct. Software allows post-deployment patches, which leads to a rapid development cycle. In contrast, hardware bugs that are found after fabrication are extremely costly to fix (and sometimes even unfixable), so the traditional hardware development cycle involves massive investment in extensive simulation and formal verification. Reconfigurable hardware, such as a Field Programmable Gate Array (FPGA), promises to propel hardware development towards an agile software-like development approach, since it enables a hardware developer to patch bugs that are detected during on-chip testing or in production. Unfortunately, FPGA programmers lack bug localization tools amenable to this rapid development cycle, since past tools mainly find bugs via simulation and verification. To develop hardware bug localization tools for a rapid development cycle, a thorough understanding of the symptoms, root causes, and fixes of hardware bugs is needed. In this paper, we first study bugs in existing FPGA designs and produce a testbed of reliably-reproducible bugs. We classify the bugs according to their intrinsic properties, symptoms, and root causes. We demonstrate that many hardware bugs are comparable to software bug counterparts, and would benefit from similar techniques for bug diagnosis and repair. Based upon our findings, we build a novel collection of hybrid static/dynamic program analysis and monitoring tools for debugging FPGA designs, showing that our tools enable a software-like development cycle by effectively reducing developers' manual efforts for bug localization. @InProceedings{ASPLOS22p946, author = {Jiacheng Ma and Gefei Zuo and Kevin Loughlin and Haoyang Zhang and Andrew Quinn and Baris Kasikci}, title = {Debugging in the Brave New World of Reconfigurable Hardware}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {946--962}, doi = {10.1145/3503222.3507701}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Mahmod, Jubayer	ASPLOS '22: "SRAM Has No Chill: Exploiting ..." SRAM Has No Chill: Exploiting Power Domain Separation to Steal On-Chip Secrets Jubayer Mahmod and Matthew Hicks (Virginia Tech, USA) The abundance of embedded systems and smart devices increases the risk of physical memory disclosure attacks. One such classic non-invasive attack exploits dynamic RAM's temperature-dependent ability to retain information across power cycles---known as a cold boot attack. When exposed to low temperatures, DRAM cells preserve their state for a short time without power, mimicking non-volatile memories in that time frame. Attackers exploit this physical phenomenon to gain access to a system's secrets, leading to data theft from encrypted storage. To prevent cold boot attacks, programmers hide secrets on-chip in Static Random-Access Memory (SRAM); by construction, on-chip SRAM is isolated from external probing and has little intrinsic capacitance, making it robust against cold boot attacks. While it is the case that SRAM protects against traditional cold boot attacks, we show that there is another way to retain information in on-chip SRAM across power cycles and software changes. This paper presents Volt Boot, an attack that demonstrates a vulnerability of on-chip volatile memories due to the physical separation common to modern system-on-chip power distribution networks. Volt Boot leverages asymmetrical power states (e.g., on vs. off) to force SRAM state retention across power cycles, eliminating the need for traditional cold boot attack enablers, such as low-temperature or intrinsic data retention time. Using several modern ARM Cortex-A devices, we demonstrate the effectiveness of the attack in caches, registers, and iRAMs. Unlike other forms of SRAM data retention attacks, Volt Boot retrieves data with 100% accuracy---without any complex post-processing. @InProceedings{ASPLOS22p1043, author = {Jubayer Mahmod and Matthew Hicks}, title = {SRAM Has No Chill: Exploiting Power Domain Separation to Steal On-Chip Secrets}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1043--1055}, doi = {10.1145/3503222.3507710}, year = {2022}, } Publisher's Version ASPLOS '22: "Invisible Bits: Hiding Secret ..." Invisible Bits: Hiding Secret Messages in SRAM’s Analog Domain Jubayer Mahmod and Matthew Hicks (Virginia Tech, USA) Electronic devices are increasingly the subject of inspection by authorities. While encryption hides secret messages, it does not hide the transmission of those secret messages---in fact, it calls attention to them. Thus, an adversary, seeing encrypted data, turns to coercion to extract the credentials required to reveal the secret message. Steganographic techniques hide secret messages in plain sight, providing the user with plausible deniability, removing the threat of coercion. This paper unveils Invisible Bits a new steganographic technique that hides secret messages in the analog domain of Static Random Access Memory (SRAM) embedded within a computing device. Unlike other memory technologies, the power-on state of SRAM reveals the analog-domain properties of its individual cells. We show how to quickly and systematically change the analog-domain properties of SRAM cells to encode data in the analog domain and how to reveal those changes by capturing SRAM's power-on state. Experiments with commercial devices show that Invisible Bits provides over 90% capacity---two orders-of-magnitude more than previous on-chip steganographic approaches, while retaining device functionality---even when the device undergoes subsequent normal operation or is shelved for months. Experiments also show that adversaries cannot differentiate between devices with encoded messages and those without. Lastly, we show how to layer encryption and error correction on top of our message encoding scheme in an end-to-end demonstration. @InProceedings{ASPLOS22p1086, author = {Jubayer Mahmod and Matthew Hicks}, title = {Invisible Bits: Hiding Secret Messages in SRAM’s Analog Domain}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1086--1098}, doi = {10.1145/3503222.3507756}, year = {2022}, } Publisher's Version
	Maleki, Saeed	ASPLOS '22: "Breaking the Computation and ..." Breaking the Computation and Communication Abstraction Barrier in Distributed Machine Learning Workloads Abhinav Jangda, Jun Huang, Guodong Liu, Amir Hossein Nodehi Sabet, Saeed Maleki, Youshan Miao, Madanlal Musuvathi, Todd Mytkowicz, and Olli Saarikivi (University of Massachusetts at Amherst, USA; Ohio State University, USA; Chinese Academy of Sciences, China; University of California at Riverside, USA; Microsoft Research, USA; Microsoft Research, China) Recent trends towards large machine learning models require both training and inference tasks to be distributed. Considering the huge cost of training these models, it is imperative to unlock optimizations in computation and communication to obtain best performance. However, the current logical separation between computation and communication kernels in machine learning frameworks misses optimization opportunities across this barrier. Breaking this abstraction can provide many optimizations to improve the performance of distributed workloads. However, manually applying these optimizations requires modifying the underlying computation and communication libraries for each scenario, which is both time consuming and error-prone. Therefore, we present CoCoNet, which contains (i) a domain specific language to express a distributed machine learning program in the form of computation and communication operations, (ii) a set of semantics preserving transformations to optimize the program, and (iii) a compiler to generate jointly optimized communication and computation GPU kernels. Providing both computation and communication as first class constructs allows users to work on a high-level abstraction and apply powerful optimizations, such as fusion or overlapping of communication and computation. CoCoNet enabled us to optimize data-, model- and pipeline-parallel workloads in large language models with only a few lines of code. Our experiments show that CoCoNet significantly outperforms state-of-the-art distributed machine learning implementations. @InProceedings{ASPLOS22p402, author = {Abhinav Jangda and Jun Huang and Guodong Liu and Amir Hossein Nodehi Sabet and Saeed Maleki and Youshan Miao and Madanlal Musuvathi and Todd Mytkowicz and Olli Saarikivi}, title = {Breaking the Computation and Communication Abstraction Barrier in Distributed Machine Learning Workloads}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {402--416}, doi = {10.1145/3503222.3507778}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional
	Malka, Gabi	ASPLOS '22: "FlexDriver: A Network Driver ..." FlexDriver: A Network Driver for Your Accelerator Haggai Eran, Maxim Fudim, Gabi Malka, Gal Shalom, Noam Cohen, Amit Hermony, Dotan Levi, Liran Liss, and Mark Silberstein (NVIDIA, Israel; Technion, Israel) We propose a new system design for connecting hardware and FPGA accelerators to the network, allowing the accelerator to directly control commodity Network Interface Cards (NICs) without using the CPU. This enables us to solve the key challenge of leveraging existing NIC hardware offloads such as virtualization, tunneling, and RDMA for accelerator networking. Our approach supports a diverse set of use cases, from direct network access for disaggregated accelerators to inline-acceleration of the network stack, all without the complex networking logic in the accelerator. To demonstrate the feasibility of this approach, we build FlexDriver (FLD), an on-accelerator hardware module that implements a NIC data-plane driver. Our main technical contribution is a mechanism that compresses the NIC control structures by two orders of magnitude, allowing FLD to achieve high networking scalability with low die area cost and no bandwidth interference with the accelerator logic. The prototype for NVIDIA Innova-2 FPGA SmartNICs showcases our design’s utility for three different accelerators: a disaggregated LTE cipher, an IP-defragmentation inline accelerator, and an IoT cryptographic-token authentication offload. These accelerators reach 25 Gbps line rate and leverage the NIC for RDMA processing, VXLAN tunneling, and traffic shaping without CPU involvement. @InProceedings{ASPLOS22p1115, author = {Haggai Eran and Maxim Fudim and Gabi Malka and Gal Shalom and Noam Cohen and Amit Hermony and Dotan Levi and Liran Liss and Mark Silberstein}, title = {FlexDriver: A Network Driver for Your Accelerator}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1115--1129}, doi = {10.1145/3503222.3507776}, year = {2022}, } Publisher's Version
	Mansouri Ghiasi, Nika	ASPLOS '22: "GenStore: A High-Performance ..." GenStore: A High-Performance In-Storage Processing System for Genome Sequence Analysis Nika Mansouri Ghiasi, Jisung Park, Harun Mustafa, Jeremie Kim, Ataberk Olgun, Arvid Gollwitzer, Damla Senol Cali, Can Firtina, Haiyu Mao, Nour Almadhoun Alserr, Rachata Ausavarungnirun, Nandita Vijaykumar, Mohammed Alser, and Onur Mutlu (ETH Zurich, Switzerland; Bionano Genomics, USA; King Mongkut's University of Technology North Bangkok, Thailand; University of Toronto, Canada) Read mapping is a fundamental step in many genomics applications. It is used to identify potential matches and differences between fragments (called reads) of a sequenced genome and an already known genome (called a reference genome). Read mapping is costly because it needs to perform approximate string matching (ASM) on large amounts of data. To address the computational challenges in genome analysis, many prior works propose various approaches such as accurate filters that select the reads within a dataset of genomic reads (called a read set) that must undergo expensive computation, efficient heuristics, and hardware acceleration. While effective at reducing the amount of expensive computation, all such approaches still require the costly movement of a large amount of data from storage to the rest of the system, which can significantly lower the end-to-end performance of read mapping in conventional and emerging genomics systems. We propose GenStore, the first in-storage processing system designed for genome sequence analysis that greatly reduces both data movement and computational overheads of genome sequence analysis by exploiting low-cost and accurate in-storage filters. GenStore leverages hardware/software co-design to address the challenges of in-storage processing, supporting reads with 1) different properties such as read lengths and error rates, which highly depend on the sequencing technology, and 2) different degrees of genetic variation compared to the reference genome, which highly depends on the genomes that are being compared. Through rigorous analysis of read mapping processes of reads with different properties and degrees of genetic variation, we meticulously design low-cost hardware accelerators and data/computation flows inside a NAND flash-based solid-state drive (SSD). Our evaluation using a wide range of real genomic datasets shows that GenStore, when implemented in three modern NAND flash-based SSDs, significantly improves the read mapping performance of state-of-the-art software (hardware) baselines by 2.07-6.05× (1.52-3.32×) for read sets with high similarity to the reference genome and 1.45-33.63× (2.70-19.2×) for read sets with low similarity to the reference genome. @InProceedings{ASPLOS22p635, author = {Nika Mansouri Ghiasi and Jisung Park and Harun Mustafa and Jeremie Kim and Ataberk Olgun and Arvid Gollwitzer and Damla Senol Cali and Can Firtina and Haiyu Mao and Nour Almadhoun Alserr and Rachata Ausavarungnirun and Nandita Vijaykumar and Mohammed Alser and Onur Mutlu}, title = {GenStore: A High-Performance In-Storage Processing System for Genome Sequence Analysis}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {635--654}, doi = {10.1145/3503222.3507702}, year = {2022}, } Publisher's Version
	Mao, Haiyu	ASPLOS '22: "GenStore: A High-Performance ..." GenStore: A High-Performance In-Storage Processing System for Genome Sequence Analysis Nika Mansouri Ghiasi, Jisung Park, Harun Mustafa, Jeremie Kim, Ataberk Olgun, Arvid Gollwitzer, Damla Senol Cali, Can Firtina, Haiyu Mao, Nour Almadhoun Alserr, Rachata Ausavarungnirun, Nandita Vijaykumar, Mohammed Alser, and Onur Mutlu (ETH Zurich, Switzerland; Bionano Genomics, USA; King Mongkut's University of Technology North Bangkok, Thailand; University of Toronto, Canada) Read mapping is a fundamental step in many genomics applications. It is used to identify potential matches and differences between fragments (called reads) of a sequenced genome and an already known genome (called a reference genome). Read mapping is costly because it needs to perform approximate string matching (ASM) on large amounts of data. To address the computational challenges in genome analysis, many prior works propose various approaches such as accurate filters that select the reads within a dataset of genomic reads (called a read set) that must undergo expensive computation, efficient heuristics, and hardware acceleration. While effective at reducing the amount of expensive computation, all such approaches still require the costly movement of a large amount of data from storage to the rest of the system, which can significantly lower the end-to-end performance of read mapping in conventional and emerging genomics systems. We propose GenStore, the first in-storage processing system designed for genome sequence analysis that greatly reduces both data movement and computational overheads of genome sequence analysis by exploiting low-cost and accurate in-storage filters. GenStore leverages hardware/software co-design to address the challenges of in-storage processing, supporting reads with 1) different properties such as read lengths and error rates, which highly depend on the sequencing technology, and 2) different degrees of genetic variation compared to the reference genome, which highly depends on the genomes that are being compared. Through rigorous analysis of read mapping processes of reads with different properties and degrees of genetic variation, we meticulously design low-cost hardware accelerators and data/computation flows inside a NAND flash-based solid-state drive (SSD). Our evaluation using a wide range of real genomic datasets shows that GenStore, when implemented in three modern NAND flash-based SSDs, significantly improves the read mapping performance of state-of-the-art software (hardware) baselines by 2.07-6.05× (1.52-3.32×) for read sets with high similarity to the reference genome and 1.45-33.63× (2.70-19.2×) for read sets with low similarity to the reference genome. @InProceedings{ASPLOS22p635, author = {Nika Mansouri Ghiasi and Jisung Park and Harun Mustafa and Jeremie Kim and Ataberk Olgun and Arvid Gollwitzer and Damla Senol Cali and Can Firtina and Haiyu Mao and Nour Almadhoun Alserr and Rachata Ausavarungnirun and Nandita Vijaykumar and Mohammed Alser and Onur Mutlu}, title = {GenStore: A High-Performance In-Storage Processing System for Genome Sequence Analysis}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {635--654}, doi = {10.1145/3503222.3507702}, year = {2022}, } Publisher's Version
	Marinov, Darko	ASPLOS '22: "Pinned Loads: Taming Speculative ..." Pinned Loads: Taming Speculative Loads in Secure Processors Zirui Neil Zhao, Houxiang Ji, Adam Morrison, Darko Marinov, and Josep Torrellas (University of Illinois at Urbana-Champaign, USA; Tel Aviv University, Israel) In security frameworks for speculative execution, an instruction is said to reach its Visibility Point (VP) when it is no longer vulnerable to pipeline squashes. Before a potentially leaky instruction reaches its VP, it has to stall—unless a defense scheme such as invisible speculation provides protection. Unfortunately, either stalling or protecting the execution of pre-VP instructions typically has a performance cost. One way to attain low-overhead safe execution is to develop techniques that speed-up the advance of the VP from older to younger instructions. In this paper, we propose one such technique. We find that the progress of the VP for loads is mostly impeded by waiting until no memory consistency violations (MCVs) are possible. Hence, our technique, called , tries to make loads invulnerable to MCVs as early as possible—a process we call pinning the loads in the pipeline. The result is faster VP progress and a reduction in the execution overhead of defense schemes. In this paper, we describe the hardware needed by , and two possible designs with different tradeoffs between hardware requirements and performance. Our evaluation shows that is very effective: extending three popular defense schemes against speculative execution attacks with reduces their average execution overhead on SPEC17 and on SPLASH2/PARSEC applications by about 50%. For example, on SPEC17, the execution overhead of the three defense schemes decreases from to , from to , and from to . @InProceedings{ASPLOS22p314, author = {Zirui Neil Zhao and Houxiang Ji and Adam Morrison and Darko Marinov and Josep Torrellas}, title = {Pinned Loads: Taming Speculative Loads in Secure Processors}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {314--328}, doi = {10.1145/3503222.3507724}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Martsenko, Kristina	ASPLOS '22: "Enzian: An Open, General, ..." Enzian: An Open, General, CPU/FPGA Platform for Systems Software Research David Cock, Abishek Ramdas, Daniel Schwyn, Michael Giardino, Adam Turowski, Zhenhao He, Nora Hossle, Dario Korolija, Melissa Licciardello, Kristina Martsenko, Reto Achermann, Gustavo Alonso, and Timothy Roscoe (ETH Zurich, Switzerland; University of British Columbia, Canada) Hybrid computing platforms, comprising CPU cores and FPGA logic, are increasingly used for accelerating data-intensive workloads in cloud deployments, and are a growing topic of interest in systems research. However, from a research perspective, existing hardware platforms are limited: they are often optimized for concrete, narrow use-cases and, therefore lack the flexibility needed to explore other applications and configurations. We show that a research group can design and build a more general, open, and affordable hardware platform for hybrid systems research. The platform, Enzian, is capable of duplicating the functionality of existing CPU/FPGA systems with comparable performance but in an open, flexible system. It couples a large FPGA with a server-class CPU in an asymmetric cache-coherent NUMA system. Enzian also enables research not possible with existing hybrid platforms, through explicit access to coherence messages, extensive thermal and power instrumentation, and an open, programmable baseboard management processor. Enzian is already being used in multiple projects, is open source (both hardware and software), and available for remote use. We present the design principles of Enzian, the challenges in building it, and evaluate it with a range of existing research use-cases alongside other, more specialized platforms, as well as demonstrating research not possible on existing platforms. @InProceedings{ASPLOS22p434, author = {David Cock and Abishek Ramdas and Daniel Schwyn and Michael Giardino and Adam Turowski and Zhenhao He and Nora Hossle and Dario Korolija and Melissa Licciardello and Kristina Martsenko and Reto Achermann and Gustavo Alonso and Timothy Roscoe}, title = {Enzian: An Open, General, CPU/FPGA Platform for Systems Software Research}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {434--451}, doi = {10.1145/3503222.3507742}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Mason, Chris	ASPLOS '22: "IOCost: Block IO Control for ..." IOCost: Block IO Control for Containers in Datacenters Tejun Heo, Dan Schatzberg, Andrew Newell, Song Liu, Saravanan Dhakshinamurthy, Iyswarya Narayanan, Josef Bacik, Chris Mason, Chunqiang Tang, and Dimitrios Skarlatos (Meta, USA; Carnegie Mellon University, USA) Resource isolation is a fundamental requirement in datacenter environments. However, our production experience in Meta’s large-scale datacenters shows that existing IO control mechanisms for block storage are inadequate in containerized environments. IO control needs to provide proportional resources to containers while taking into account the hardware heterogeneity of storage devices and the idiosyncrasies of the workloads deployed in datacenters. The speed of modern SSDs requires IO control to execute with low-overheads. Furthermore, IO control should strive for work conservation, take into account the interactions with the memory management subsystem, and avoid priority inversions that lead to isolation failures. To address these challenges, this paper presents IOCost, an IO control solution that is designed for containerized environments and provides scalable, work-conserving, and low-overhead IO control for heterogeneous storage devices and diverse workloads in datacenters. IOCost performs offline profiling to build a device model and uses it to estimate device occupancy of each IO request. To minimize runtime overhead, it separates IO control into a fast per-IO issue path and a slower periodic planning path. A novel work-conserving budget donation algorithm enables containers to dynamically share unused budget. We have deployed IOCost across the entirety of Meta’s datacenters comprised of millions of ma- chines, upstreamed IOCost to the Linux kernel, and open-sourced our device-profiling tools. IOCost has been running in production for two years, providing IO control for Meta’s fleet. We describe the design of IOCost and share our experience deploying it at scale. @InProceedings{ASPLOS22p595, author = {Tejun Heo and Dan Schatzberg and Andrew Newell and Song Liu and Saravanan Dhakshinamurthy and Iyswarya Narayanan and Josef Bacik and Chris Mason and Chunqiang Tang and Dimitrios Skarlatos}, title = {IOCost: Block IO Control for Containers in Datacenters}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {595--608}, doi = {10.1145/3503222.3507727}, year = {2022}, } Publisher's Version
	Mathur, Umang	ASPLOS '22: "A Tree Clock Data Structure ..." A Tree Clock Data Structure for Causal Orderings in Concurrent Executions Umang Mathur, Andreas Pavlogiannis, Hünkar Can Tunç, and Mahesh Viswanathan (National University of Singapore, Singapore; Aarhus University, Denmark; University of Illinois at Urbana-Champaign, USA) Dynamic techniques are a scalable and effective way to analyze concurrent programs. Instead of analyzing all behaviors of a program, these techniques detect errors by focusing on a single program execution. Often a crucial step in these techniques is to define a causal ordering between events in the execution, which is then computed using vector clocks, a simple data structure that stores logical times of threads. The two basic operations of vector clocks, namely join and copy, require Θ(k) time, where k is the number of threads. Thus they are a computational bottleneck when k is large. In this work, we introduce tree clocks, a new data structure that replaces vector clocks for computing causal orderings in program executions. Joining and copying tree clocks takes time that is roughly proportional to the number of entries being modified, and hence the two operations do not suffer the a-priori Θ(k) cost per application. We show that when used to compute the classic happens-before (HB) partial order, tree clocks are optimal, in the sense that no other data structure can lead to smaller asymptotic running time. Moreover, we demonstrate that tree clocks can be used to compute other partial orders, such as schedulable-happens-before (SHB) and the standard Mazurkiewicz (MAZ) partial order, and thus are a versatile data structure. Our experiments show that just by replacing vector clocks with tree clocks, the computation becomes from 2.02 × faster (MAZ) to 2.66 × (SHB) and 2.97 × (HB) on average per benchmark. These results illustrate that tree clocks have the potential to become a standard data structure with wide applications in concurrent analyses. @InProceedings{ASPLOS22p710, author = {Umang Mathur and Andreas Pavlogiannis and Hünkar Can Tunç and Mahesh Viswanathan}, title = {A Tree Clock Data Structure for Causal Orderings in Concurrent Executions}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {710--725}, doi = {10.1145/3503222.3507734}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Mellor-Crummey, John	ASPLOS '22: "ValueExpert: Exploring Value ..." ValueExpert: Exploring Value Patterns in GPU-Accelerated Applications Keren Zhou, Yueming Hao, John Mellor-Crummey, Xiaozhu Meng, and Xu Liu (Rice University, USA; North Carolina State University, USA) General-purpose GPUs have become common in modern computing systems to accelerate applications in many domains, including machine learning, high-performance computing, and autonomous driving. However, inefficiencies abound in GPU-accelerated applications, which prevent them from obtaining bare-metal performance. Performance tools play an important role in understanding performance inefficiencies in complex code bases. Many GPU performance tools pinpoint time-consuming code and provide high-level performance insights but overlook one important performance issue---value-related inefficiencies, which exist in many GPU code bases. In this paper, we present ValueExpert, a novel tool to pinpoint value-related inefficiencies in GPU applications. ValueExpert monitors application execution to capture values produced and used by each load and store operation in GPU kernels, recognizes multiple value patterns, and provides intuitive optimization guidance. We address systemic challenges in collecting, maintaining, and analyzing voluminous performance data from many GPU threads to make ValueExpert applicable to complex applications. We evaluate ValueExpert on a wide range of well-tuned benchmarks and applications, including PyTorch, Darknet, LAMMPS, Castro, and many others. ValueExpert is able to identify previously unknown performance issues and provide suggestions for nontrivial performance improvements with typically less than five lines of code changes. We verify our optimizations with application developers and upstream fixes to their repositories. @InProceedings{ASPLOS22p171, author = {Keren Zhou and Yueming Hao and John Mellor-Crummey and Xiaozhu Meng and Xu Liu}, title = {ValueExpert: Exploring Value Patterns in GPU-Accelerated Applications}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {171--185}, doi = {10.1145/3503222.3507708}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Meng, Xiaozhu	ASPLOS '22: "ValueExpert: Exploring Value ..." ValueExpert: Exploring Value Patterns in GPU-Accelerated Applications Keren Zhou, Yueming Hao, John Mellor-Crummey, Xiaozhu Meng, and Xu Liu (Rice University, USA; North Carolina State University, USA) General-purpose GPUs have become common in modern computing systems to accelerate applications in many domains, including machine learning, high-performance computing, and autonomous driving. However, inefficiencies abound in GPU-accelerated applications, which prevent them from obtaining bare-metal performance. Performance tools play an important role in understanding performance inefficiencies in complex code bases. Many GPU performance tools pinpoint time-consuming code and provide high-level performance insights but overlook one important performance issue---value-related inefficiencies, which exist in many GPU code bases. In this paper, we present ValueExpert, a novel tool to pinpoint value-related inefficiencies in GPU applications. ValueExpert monitors application execution to capture values produced and used by each load and store operation in GPU kernels, recognizes multiple value patterns, and provides intuitive optimization guidance. We address systemic challenges in collecting, maintaining, and analyzing voluminous performance data from many GPU threads to make ValueExpert applicable to complex applications. We evaluate ValueExpert on a wide range of well-tuned benchmarks and applications, including PyTorch, Darknet, LAMMPS, Castro, and many others. ValueExpert is able to identify previously unknown performance issues and provide suggestions for nontrivial performance improvements with typically less than five lines of code changes. We verify our optimizations with application developers and upstream fixes to their repositories. @InProceedings{ASPLOS22p171, author = {Keren Zhou and Yueming Hao and John Mellor-Crummey and Xiaozhu Meng and Xu Liu}, title = {ValueExpert: Exploring Value Patterns in GPU-Accelerated Applications}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {171--185}, doi = {10.1145/3503222.3507708}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Merczynski-Hait, Andrew	ASPLOS '22: "PLD: Fast FPGA Compilation ..." PLD: Fast FPGA Compilation to Make Reconfigurable Acceleration Compatible with Modern Incremental Refinement Software Development Yuanlong Xiao, Eric Micallef, Andrew Butt, Matthew Hofmann, Marc Alston, Matthew Goldsmith, Andrew Merczynski-Hait, and André DeHon (University of Pennsylvania, USA) FPGA-based accelerators are demonstrating significant absolute performance and energy efficiency compared with general-purpose CPUs. While FPGA computations can now be described in standard, programming languages, like C, development for FPGAs accelerators remains tedious and inaccessible to modern software engineers. Slow compiles (potentially taking tens of hours) inhibit the rapid, incremental refinement of designs that is the hallmark of modern software engineering. To address this issue, we introduce separate compilation and linkage into the FPGA design flow, providing faster design turns more familiar to software development. To realize this flow, we provide abstractions, compiler options, and compiler flow that allow the same C source code to be compiled to processor cores in seconds and to FPGA regions in minutes, providing the missing -O0 and -O1 options familiar in software development. This raises the FPGA programming level and standardizes the programming experience, bringing FPGA-based accelerators into a more familiar software platform ecosystem for software engineers. @InProceedings{ASPLOS22p933, author = {Yuanlong Xiao and Eric Micallef and Andrew Butt and Matthew Hofmann and Marc Alston and Matthew Goldsmith and Andrew Merczynski-Hait and André DeHon}, title = {PLD: Fast FPGA Compilation to Make Reconfigurable Acceleration Compatible with Modern Incremental Refinement Software Development}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {933--945}, doi = {10.1145/3503222.3507740}, year = {2022}, } Publisher's Version
	Miano, Sebastiano	ASPLOS '22: "Domain Specific Run Time Optimization ..." Domain Specific Run Time Optimization for Software Data Planes Sebastiano Miano, Alireza Sanaee, Fulvio Risso, Gábor Rétvári, and Gianni Antichi (Queen Mary University of London, UK; Politecnico di Torino, Italy; Budapest University of Technology and Economics, Hungary) State-of-the-art approaches to design, develop and optimize software packet-processing programs are based on static compilation: the compiler's input is a description of the forwarding plane semantics and the output is a binary that can accommodate any control plane configuration or input traffic. In this paper, we demonstrate that tracking control plane actions and packet-level traffic dynamics at run time opens up new opportunities for code specialization. We present Morpheus, a system working alongside static compilers that continuously optimizes the targeted networking code. We introduce a number of new techniques, from static code analysis to adaptive code instrumentation, and we implement a toolbox of domain specific optimizations that are not restricted to a specific data plane framework or programming language. We apply Morpheus to several eBPF and DPDK programs including Katran, Facebook's production-grade load balancer. We compare Morpheus against state-of-the-art optimization frameworks and show that it can bring up to 2x throughput improvement, while halving the 99th percentile latency. @InProceedings{ASPLOS22p1148, author = {Sebastiano Miano and Alireza Sanaee and Fulvio Risso and Gábor Rétvári and Gianni Antichi}, title = {Domain Specific Run Time Optimization for Software Data Planes}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1148--1164}, doi = {10.1145/3503222.3507769}, year = {2022}, } Publisher's Version Published Artifact Info Artifacts Available Artifacts Functional Results Reproduced
	Miao, Youshan	ASPLOS '22: "Breaking the Computation and ..." Breaking the Computation and Communication Abstraction Barrier in Distributed Machine Learning Workloads Abhinav Jangda, Jun Huang, Guodong Liu, Amir Hossein Nodehi Sabet, Saeed Maleki, Youshan Miao, Madanlal Musuvathi, Todd Mytkowicz, and Olli Saarikivi (University of Massachusetts at Amherst, USA; Ohio State University, USA; Chinese Academy of Sciences, China; University of California at Riverside, USA; Microsoft Research, USA; Microsoft Research, China) Recent trends towards large machine learning models require both training and inference tasks to be distributed. Considering the huge cost of training these models, it is imperative to unlock optimizations in computation and communication to obtain best performance. However, the current logical separation between computation and communication kernels in machine learning frameworks misses optimization opportunities across this barrier. Breaking this abstraction can provide many optimizations to improve the performance of distributed workloads. However, manually applying these optimizations requires modifying the underlying computation and communication libraries for each scenario, which is both time consuming and error-prone. Therefore, we present CoCoNet, which contains (i) a domain specific language to express a distributed machine learning program in the form of computation and communication operations, (ii) a set of semantics preserving transformations to optimize the program, and (iii) a compiler to generate jointly optimized communication and computation GPU kernels. Providing both computation and communication as first class constructs allows users to work on a high-level abstraction and apply powerful optimizations, such as fusion or overlapping of communication and computation. CoCoNet enabled us to optimize data-, model- and pipeline-parallel workloads in large language models with only a few lines of code. Our experiments show that CoCoNet significantly outperforms state-of-the-art distributed machine learning implementations. @InProceedings{ASPLOS22p402, author = {Abhinav Jangda and Jun Huang and Guodong Liu and Amir Hossein Nodehi Sabet and Saeed Maleki and Youshan Miao and Madanlal Musuvathi and Todd Mytkowicz and Olli Saarikivi}, title = {Breaking the Computation and Communication Abstraction Barrier in Distributed Machine Learning Workloads}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {402--416}, doi = {10.1145/3503222.3507778}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional
	Micallef, Eric	ASPLOS '22: "PLD: Fast FPGA Compilation ..." PLD: Fast FPGA Compilation to Make Reconfigurable Acceleration Compatible with Modern Incremental Refinement Software Development Yuanlong Xiao, Eric Micallef, Andrew Butt, Matthew Hofmann, Marc Alston, Matthew Goldsmith, Andrew Merczynski-Hait, and André DeHon (University of Pennsylvania, USA) FPGA-based accelerators are demonstrating significant absolute performance and energy efficiency compared with general-purpose CPUs. While FPGA computations can now be described in standard, programming languages, like C, development for FPGAs accelerators remains tedious and inaccessible to modern software engineers. Slow compiles (potentially taking tens of hours) inhibit the rapid, incremental refinement of designs that is the hallmark of modern software engineering. To address this issue, we introduce separate compilation and linkage into the FPGA design flow, providing faster design turns more familiar to software development. To realize this flow, we provide abstractions, compiler options, and compiler flow that allow the same C source code to be compiled to processor cores in seconds and to FPGA regions in minutes, providing the missing -O0 and -O1 options familiar in software development. This raises the FPGA programming level and standardizes the programming experience, bringing FPGA-based accelerators into a more familiar software platform ecosystem for software engineers. @InProceedings{ASPLOS22p933, author = {Yuanlong Xiao and Eric Micallef and Andrew Butt and Matthew Hofmann and Marc Alston and Matthew Goldsmith and Andrew Merczynski-Hait and André DeHon}, title = {PLD: Fast FPGA Compilation to Make Reconfigurable Acceleration Compatible with Modern Incremental Refinement Software Development}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {933--945}, doi = {10.1145/3503222.3507740}, year = {2022}, } Publisher's Version
	Michelogiannakis, George	ASPLOS '22: "Temporal and SFQ Pulse-Streams ..." Temporal and SFQ Pulse-Streams Encoding for Area-Efficient Superconducting Accelerators Patricia Gonzalez-Guerrero, Meriam Gay Bautista, Darren Lyles, and George Michelogiannakis (Lawrence Berkeley National Laboratory, USA) Superconducting technology is a prime candidate for the future of computing. However, current superconducting prototypes are limited to small-scale examples due to stringent area constraints and complex architectures inspired from voltage-level encoding in CMOS; this is at odds with the ps-wide Single Quantum Flux (SFQ) pulses used in superconductors to carry information. In this work, we propose a wave-pipelined Unary SFQ (U-SFQ) architecture that leverages the advantages of two data representations: pulse-streams and Race Logic (RL). We introduce novel building blocks such as multipliers, adders, and memory cells, which leverage the natural properties of SFQ pulses to mitigate area constraints. We then design and simulate three popular hardware accelerators: i) a Processing Element (PE), typically used in spatial architectures; ii) A dot-product-unit (DPU), one of the most popular accelerators in artificial neural networks and digital signal processing (DSP); and iii) A Finite Impulse Response (FIR) filter, a popular and computationally demanding DSP accelerator. The proposed U-SFQ building blocks require up to 200× fewer JJs compared to their SFQ binary counterparts, exposing an area-delay trade-off. This work mitigates the stringent area constraints of superconducting technology. @InProceedings{ASPLOS22p963, author = {Patricia Gonzalez-Guerrero and Meriam Gay Bautista and Darren Lyles and George Michelogiannakis}, title = {Temporal and SFQ Pulse-Streams Encoding for Area-Efficient Superconducting Accelerators}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {963--976}, doi = {10.1145/3503222.3507765}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available
	Miemietz, Till	ASPLOS '22: "Efficient and Scalable Core ..." Efficient and Scalable Core Multiplexing with M³v Nils Asmussen, Sebastian Haas, Carsten Weinhold, Till Miemietz, and Michael Roitzsch (Barkhausen Institut, Germany) The M³ system (ASPLOS ’16) proposed a hardware/software co-design that simplifies integration between general-purpose cores and special-purpose accelerators, allowing users to easily utilize them in a unified manner. M³ is a tiled architecture, whose tiles (cores and accelerators) are partitioned between applications, such that each tile is dedicated to its own application. The M³x system (ATC ’19) extended M³ by trading off some isolation to enable coarse-grained multiplexing of tiles among multiple applications. With M³x, if source tile t₁ runs code of application p and sends a message m to destination tile t₂ while t₂ is currently not associated with p, then m is forwarded to the right place through a “slow path”, via some special OS tile. In this paper, we present M³v, which extends M³x by further trading off some isolation between applications to support “fast path” communication that does not require the said OS tile’s involvement. Thus, with M³v, a tile can be efficiently multiplexed between applications provided it is a general-purpose core. M³v achieves this goal by 1) adding a local multiplexer to each such core, and by 2) virtualizing the core’s hardware component responsible for cross-tile communications. We prototype M³v using RISC-V cores on an FPGA platform and show that it significantly outperforms M³x and may achieve competitive performance to Linux. @InProceedings{ASPLOS22p452, author = {Nils Asmussen and Sebastian Haas and Carsten Weinhold and Till Miemietz and Michael Roitzsch}, title = {Efficient and Scalable Core Multiplexing with M³v}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {452--466}, doi = {10.1145/3503222.3507741}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Min, Dongmoon	ASPLOS '22: "CryoWire: Wire-Driven Microarchitecture ..." CryoWire: Wire-Driven Microarchitecture Designs for Cryogenic Computing Dongmoon Min, Yujin Chung, Ilkwon Byun, Junpyo Kim, and Jangwoo Kim (Seoul National University, South Korea) Cryogenic computing, which runs a computer device at an extremely low temperature, is promising thanks to its significant reduction of wire resistance as well as leakage current. Recent studies on cryogenic computing have focused on various architectural units including the main memory, cache, and CPU core running at 77K. However, little research has been conducted to fully exploit the fast cryogenic wires, even though the slow wires are becoming more serious performance bottleneck in modern processors. In this paper, we propose a CPU microarchitecture which extensively exploits the fast wires at 77K. For this goal, we first introduce our validated cryogenic-performance models for the CPU pipeline and network on chip (NoC), whose performance can be significantly limited by the slow wires. Next, based on the analysis with the models, we architect CryoSP and CryoBus as our pipeline and NoC designs to fully exploit the fast wires. Our evaluation shows that our cryogenic computer equipped with both microarchitectures achieves 3.82 times higher system-level performance compared to the conventional computer system thanks to the 96% higher clock frequency of CryoSP and five times lower NoC latency of CryoBus. @InProceedings{ASPLOS22p903, author = {Dongmoon Min and Yujin Chung and Ilkwon Byun and Junpyo Kim and Jangwoo Kim}, title = {CryoWire: Wire-Driven Microarchitecture Designs for Cryogenic Computing}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {903--917}, doi = {10.1145/3503222.3507749}, year = {2022}, } Publisher's Version
	Ming, Jiang	ASPLOS '22: "One Size Does Not Fit All: ..." One Size Does Not Fit All: Security Hardening of MIPS Embedded Systems via Static Binary Debloating for Shared Libraries Haotian Zhang, Mengfei Ren, Yu Lei, and Jiang Ming (University of Texas at Arlington, USA) Embedded systems have become prominent targets for cyberattacks. To exploit firmware’s memory corruption vulnerabilities, cybercriminals harvest reusable code gadgets from the large shared library codebase (e.g., uClibc). Unfortunately, unlike their desktop counterparts, embedded systems lack essential computing resources to enforce security hardening techniques. Recently, we have witnessed a surge of software debloating as a new defense mechanism against code-reuse attacks; it erases unused code to significantly diminish the possibilities of constructing reusable gadgets. Because of the single firmware image update style, static library debloating shows promise to fortify embedded systems without compromising performance and forward compatibility. However, static library debloating on stripped binaries (e.g., firmware’s shared libraries) is still an enormous challenge. In this paper, we show that this challenge is not insurmountable for MIPS firmware. We develop a novel system, named uTrimmer, to identify and wipe out unused basic blocks from shared libraries’ binary code, without causing additional runtime overhead or memory consumption. We propose a new method to identify address-taken blocks/functions, which further help us maintain an inter-procedural control flow graph to conservatively include library code that could be potentially used by firmware. By capturing address access patterns for position-independent code, we circumvent the challenge of determining code-pointer targets and safely eliminate unused code. We run uTrimmer to debloat shared libraries for SPEC CPU2017 benchmarks, popular firmware applications (e.g., Apache, BusyBox, and OpenSSL), and a real-world wireless router firmware image. Our experiments show that not only does uTrimmer deliver functional programs, but also it can cut the exposed code surface and eliminate various reusable code gadgets remarkably. uTrimmer’s debloating capability can compete with the static linking results. @InProceedings{ASPLOS22p255, author = {Haotian Zhang and Mengfei Ren and Yu Lei and Jiang Ming}, title = {One Size Does Not Fit All: Security Hardening of MIPS Embedded Systems via Static Binary Debloating for Shared Libraries}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {255--270}, doi = {10.1145/3503222.3507768}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Mirhoseini, Azalia	ASPLOS '22: "A Full-Stack Search Technique ..." A Full-Stack Search Technique for Domain Optimized Deep Learning Accelerators Dan Zhang, Safeen Huda, Ebrahim Songhori, Kartik Prabhu, Quoc Le, Anna Goldie, and Azalia Mirhoseini (Google Brain, USA; Google, USA; Stanford University, USA) The rapidly-changing deep learning landscape presents a unique opportunity for building inference accelerators optimized for specific datacenter-scale workloads. We propose Full-stack Accelerator Search Technique (FAST), a hardware accelerator search framework that defines a broad optimization environment covering key design decisions within the hardware-software stack, including hardware datapath, software scheduling, and compiler passes such as operation fusion and tensor padding. In this paper, we analyze bottlenecks in state-of-the-art vision and natural language processing (NLP) models, including EfficientNet and BERT, and use FAST to design accelerators capable of addressing these bottlenecks. FAST-generated accelerators optimized for single workloads improve Perf/TDP by 3.7× on average across all benchmarks compared to TPU-v3. A FAST-generated accelerator optimized for serving a suite of workloads improves Perf/TDP by 2.4× on average compared to TPU-v3. Our return on investment analysis shows that FAST-generated accelerators can potentially be practical for moderate-sized datacenter deployments. @InProceedings{ASPLOS22p27, author = {Dan Zhang and Safeen Huda and Ebrahim Songhori and Kartik Prabhu and Quoc Le and Anna Goldie and Azalia Mirhoseini}, title = {A Full-Stack Search Technique for Domain Optimized Deep Learning Accelerators}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {27--42}, doi = {10.1145/3503222.3507767}, year = {2022}, } Publisher's Version
	Mitra, Tulika	ASPLOS '22: "REVAMP: A Systematic Framework ..." REVAMP: A Systematic Framework for Heterogeneous CGRA Realization Thilini Kaushalya Bandara, Dhananjaya Wijerathne, Tulika Mitra, and Li-Shiuan Peh (National University of Singapore, Singapore) Coarse-Grained Reconfigurable Architectures (CGRAs) provide an excellent balance between performance, energy efficiency, and flexibility. However, increasingly sophisticated applications, especially on the edge devices, demand even better energy efficiency for longer battery life. Most CGRAs adhere to a canonical structure where a homogeneous set of processing elements and memories communicate through a regular interconnect due to the simplicity of the design. Unfortunately, the homogeneity leads to substantial idle resources while mapping irregular applications and creates inefficiency. We plan to mitigate the inefficiency by systematically and judiciously introducing heterogeneity in CGRAs in tandem with appropriate compiler support. We propose REVAMP, an automated design space exploration framework that helps architects uncover and add pertinent heterogeneity to a diverse range of originally homogeneous CGRAs when fed with a suite of target applications. REVAMP explores a comprehensive set of optimizations encompassing compute, network, and memory heterogeneity, thereby converting a uniform CGRA into a more irregular architecture with improved energy efficiency. As CGRAs are inherently software scheduled, any micro-architectural optimizations need to be partnered with corresponding compiler support, which is challenging with heterogeneity. The REVAMP framework extends compiler support for efficient mapping of loop kernels on the derived heterogeneous CGRA architectures. We showcase REVAMP on three state-of-the-art homogeneous CGRAs, demonstrating how REVAMP derives a heterogeneous variant of each homogeneous architecture, with its corresponding compiler optimizations. Our results show that the derived heterogeneous architectures achieve up to 52.4% power reduction, 38.1% area reduction, and 36% average energy reduction over the corresponding homogeneous versions with minimal performance impact for the selected kernel suite. @InProceedings{ASPLOS22p918, author = {Thilini Kaushalya Bandara and Dhananjaya Wijerathne and Tulika Mitra and Li-Shiuan Peh}, title = {REVAMP: A Systematic Framework for Heterogeneous CGRA Realization}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {918--932}, doi = {10.1145/3503222.3507772}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Morrison, Adam	ASPLOS '22: "The Benefits of General-Purpose ..." The Benefits of General-Purpose On-NIC Memory Boris Pismenny, Liran Liss, Adam Morrison, and Dan Tsafrir (Technion, Israel; NVIDIA, Israel; Tel Aviv University, Israel; VMware Research, Israel) We propose to use the small, newly available on-NIC memory ("nicmem") to keep pace with the rapidly increasing performance of NICs. We motivate our proposal by accelerating two types of workload classes: NFV and key-value stores. As NFV workloads frequently operate on headers---rather than data---of incoming packets, we introduce a new packet-processing architecture that splits between the two, keeping the data on nicmem when possible and thus reducing PCIe traffic, memory bandwidth, and CPU processing time. Our approach consequently shortens NFV latency by up to 23% and increases its throughput by up to 19%. Similarly, because key-value stores commonly exhibit skewed distributions, we introduce a new network stack mechanism that lets applications keep frequently accessed items on nicmem. Our design shortens memcached latency by up to 43% and increases its throughput by up to 80%. @InProceedings{ASPLOS22p1130, author = {Boris Pismenny and Liran Liss and Adam Morrison and Dan Tsafrir}, title = {The Benefits of General-Purpose On-NIC Memory}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1130--1147}, doi = {10.1145/3503222.3507711}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced ASPLOS '22: "Pinned Loads: Taming Speculative ..." Pinned Loads: Taming Speculative Loads in Secure Processors Zirui Neil Zhao, Houxiang Ji, Adam Morrison, Darko Marinov, and Josep Torrellas (University of Illinois at Urbana-Champaign, USA; Tel Aviv University, Israel) In security frameworks for speculative execution, an instruction is said to reach its Visibility Point (VP) when it is no longer vulnerable to pipeline squashes. Before a potentially leaky instruction reaches its VP, it has to stall—unless a defense scheme such as invisible speculation provides protection. Unfortunately, either stalling or protecting the execution of pre-VP instructions typically has a performance cost. One way to attain low-overhead safe execution is to develop techniques that speed-up the advance of the VP from older to younger instructions. In this paper, we propose one such technique. We find that the progress of the VP for loads is mostly impeded by waiting until no memory consistency violations (MCVs) are possible. Hence, our technique, called , tries to make loads invulnerable to MCVs as early as possible—a process we call pinning the loads in the pipeline. The result is faster VP progress and a reduction in the execution overhead of defense schemes. In this paper, we describe the hardware needed by , and two possible designs with different tradeoffs between hardware requirements and performance. Our evaluation shows that is very effective: extending three popular defense schemes against speculative execution attacks with reduces their average execution overhead on SPEC17 and on SPLASH2/PARSEC applications by about 50%. For example, on SPEC17, the execution overhead of the three defense schemes decreases from to , from to , and from to . @InProceedings{ASPLOS22p314, author = {Zirui Neil Zhao and Houxiang Ji and Adam Morrison and Darko Marinov and Josep Torrellas}, title = {Pinned Loads: Taming Speculative Loads in Secure Processors}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {314--328}, doi = {10.1145/3503222.3507724}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Mustafa, Harun	ASPLOS '22: "GenStore: A High-Performance ..." GenStore: A High-Performance In-Storage Processing System for Genome Sequence Analysis Nika Mansouri Ghiasi, Jisung Park, Harun Mustafa, Jeremie Kim, Ataberk Olgun, Arvid Gollwitzer, Damla Senol Cali, Can Firtina, Haiyu Mao, Nour Almadhoun Alserr, Rachata Ausavarungnirun, Nandita Vijaykumar, Mohammed Alser, and Onur Mutlu (ETH Zurich, Switzerland; Bionano Genomics, USA; King Mongkut's University of Technology North Bangkok, Thailand; University of Toronto, Canada) Read mapping is a fundamental step in many genomics applications. It is used to identify potential matches and differences between fragments (called reads) of a sequenced genome and an already known genome (called a reference genome). Read mapping is costly because it needs to perform approximate string matching (ASM) on large amounts of data. To address the computational challenges in genome analysis, many prior works propose various approaches such as accurate filters that select the reads within a dataset of genomic reads (called a read set) that must undergo expensive computation, efficient heuristics, and hardware acceleration. While effective at reducing the amount of expensive computation, all such approaches still require the costly movement of a large amount of data from storage to the rest of the system, which can significantly lower the end-to-end performance of read mapping in conventional and emerging genomics systems. We propose GenStore, the first in-storage processing system designed for genome sequence analysis that greatly reduces both data movement and computational overheads of genome sequence analysis by exploiting low-cost and accurate in-storage filters. GenStore leverages hardware/software co-design to address the challenges of in-storage processing, supporting reads with 1) different properties such as read lengths and error rates, which highly depend on the sequencing technology, and 2) different degrees of genetic variation compared to the reference genome, which highly depends on the genomes that are being compared. Through rigorous analysis of read mapping processes of reads with different properties and degrees of genetic variation, we meticulously design low-cost hardware accelerators and data/computation flows inside a NAND flash-based solid-state drive (SSD). Our evaluation using a wide range of real genomic datasets shows that GenStore, when implemented in three modern NAND flash-based SSDs, significantly improves the read mapping performance of state-of-the-art software (hardware) baselines by 2.07-6.05× (1.52-3.32×) for read sets with high similarity to the reference genome and 1.45-33.63× (2.70-19.2×) for read sets with low similarity to the reference genome. @InProceedings{ASPLOS22p635, author = {Nika Mansouri Ghiasi and Jisung Park and Harun Mustafa and Jeremie Kim and Ataberk Olgun and Arvid Gollwitzer and Damla Senol Cali and Can Firtina and Haiyu Mao and Nour Almadhoun Alserr and Rachata Ausavarungnirun and Nandita Vijaykumar and Mohammed Alser and Onur Mutlu}, title = {GenStore: A High-Performance In-Storage Processing System for Genome Sequence Analysis}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {635--654}, doi = {10.1145/3503222.3507702}, year = {2022}, } Publisher's Version
	Musuvathi, Madanlal	ASPLOS '22: "Breaking the Computation and ..." Breaking the Computation and Communication Abstraction Barrier in Distributed Machine Learning Workloads Abhinav Jangda, Jun Huang, Guodong Liu, Amir Hossein Nodehi Sabet, Saeed Maleki, Youshan Miao, Madanlal Musuvathi, Todd Mytkowicz, and Olli Saarikivi (University of Massachusetts at Amherst, USA; Ohio State University, USA; Chinese Academy of Sciences, China; University of California at Riverside, USA; Microsoft Research, USA; Microsoft Research, China) Recent trends towards large machine learning models require both training and inference tasks to be distributed. Considering the huge cost of training these models, it is imperative to unlock optimizations in computation and communication to obtain best performance. However, the current logical separation between computation and communication kernels in machine learning frameworks misses optimization opportunities across this barrier. Breaking this abstraction can provide many optimizations to improve the performance of distributed workloads. However, manually applying these optimizations requires modifying the underlying computation and communication libraries for each scenario, which is both time consuming and error-prone. Therefore, we present CoCoNet, which contains (i) a domain specific language to express a distributed machine learning program in the form of computation and communication operations, (ii) a set of semantics preserving transformations to optimize the program, and (iii) a compiler to generate jointly optimized communication and computation GPU kernels. Providing both computation and communication as first class constructs allows users to work on a high-level abstraction and apply powerful optimizations, such as fusion or overlapping of communication and computation. CoCoNet enabled us to optimize data-, model- and pipeline-parallel workloads in large language models with only a few lines of code. Our experiments show that CoCoNet significantly outperforms state-of-the-art distributed machine learning implementations. @InProceedings{ASPLOS22p402, author = {Abhinav Jangda and Jun Huang and Guodong Liu and Amir Hossein Nodehi Sabet and Saeed Maleki and Youshan Miao and Madanlal Musuvathi and Todd Mytkowicz and Olli Saarikivi}, title = {Breaking the Computation and Communication Abstraction Barrier in Distributed Machine Learning Workloads}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {402--416}, doi = {10.1145/3503222.3507778}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional
	Mutlu, Onur	ASPLOS '22: "GenStore: A High-Performance ..." GenStore: A High-Performance In-Storage Processing System for Genome Sequence Analysis Nika Mansouri Ghiasi, Jisung Park, Harun Mustafa, Jeremie Kim, Ataberk Olgun, Arvid Gollwitzer, Damla Senol Cali, Can Firtina, Haiyu Mao, Nour Almadhoun Alserr, Rachata Ausavarungnirun, Nandita Vijaykumar, Mohammed Alser, and Onur Mutlu (ETH Zurich, Switzerland; Bionano Genomics, USA; King Mongkut's University of Technology North Bangkok, Thailand; University of Toronto, Canada) Read mapping is a fundamental step in many genomics applications. It is used to identify potential matches and differences between fragments (called reads) of a sequenced genome and an already known genome (called a reference genome). Read mapping is costly because it needs to perform approximate string matching (ASM) on large amounts of data. To address the computational challenges in genome analysis, many prior works propose various approaches such as accurate filters that select the reads within a dataset of genomic reads (called a read set) that must undergo expensive computation, efficient heuristics, and hardware acceleration. While effective at reducing the amount of expensive computation, all such approaches still require the costly movement of a large amount of data from storage to the rest of the system, which can significantly lower the end-to-end performance of read mapping in conventional and emerging genomics systems. We propose GenStore, the first in-storage processing system designed for genome sequence analysis that greatly reduces both data movement and computational overheads of genome sequence analysis by exploiting low-cost and accurate in-storage filters. GenStore leverages hardware/software co-design to address the challenges of in-storage processing, supporting reads with 1) different properties such as read lengths and error rates, which highly depend on the sequencing technology, and 2) different degrees of genetic variation compared to the reference genome, which highly depends on the genomes that are being compared. Through rigorous analysis of read mapping processes of reads with different properties and degrees of genetic variation, we meticulously design low-cost hardware accelerators and data/computation flows inside a NAND flash-based solid-state drive (SSD). Our evaluation using a wide range of real genomic datasets shows that GenStore, when implemented in three modern NAND flash-based SSDs, significantly improves the read mapping performance of state-of-the-art software (hardware) baselines by 2.07-6.05× (1.52-3.32×) for read sets with high similarity to the reference genome and 1.45-33.63× (2.70-19.2×) for read sets with low similarity to the reference genome. @InProceedings{ASPLOS22p635, author = {Nika Mansouri Ghiasi and Jisung Park and Harun Mustafa and Jeremie Kim and Ataberk Olgun and Arvid Gollwitzer and Damla Senol Cali and Can Firtina and Haiyu Mao and Nour Almadhoun Alserr and Rachata Ausavarungnirun and Nandita Vijaykumar and Mohammed Alser and Onur Mutlu}, title = {GenStore: A High-Performance In-Storage Processing System for Genome Sequence Analysis}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {635--654}, doi = {10.1145/3503222.3507702}, year = {2022}, } Publisher's Version
	Mytkowicz, Todd	ASPLOS '22: "Breaking the Computation and ..." Breaking the Computation and Communication Abstraction Barrier in Distributed Machine Learning Workloads Abhinav Jangda, Jun Huang, Guodong Liu, Amir Hossein Nodehi Sabet, Saeed Maleki, Youshan Miao, Madanlal Musuvathi, Todd Mytkowicz, and Olli Saarikivi (University of Massachusetts at Amherst, USA; Ohio State University, USA; Chinese Academy of Sciences, China; University of California at Riverside, USA; Microsoft Research, USA; Microsoft Research, China) Recent trends towards large machine learning models require both training and inference tasks to be distributed. Considering the huge cost of training these models, it is imperative to unlock optimizations in computation and communication to obtain best performance. However, the current logical separation between computation and communication kernels in machine learning frameworks misses optimization opportunities across this barrier. Breaking this abstraction can provide many optimizations to improve the performance of distributed workloads. However, manually applying these optimizations requires modifying the underlying computation and communication libraries for each scenario, which is both time consuming and error-prone. Therefore, we present CoCoNet, which contains (i) a domain specific language to express a distributed machine learning program in the form of computation and communication operations, (ii) a set of semantics preserving transformations to optimize the program, and (iii) a compiler to generate jointly optimized communication and computation GPU kernels. Providing both computation and communication as first class constructs allows users to work on a high-level abstraction and apply powerful optimizations, such as fusion or overlapping of communication and computation. CoCoNet enabled us to optimize data-, model- and pipeline-parallel workloads in large language models with only a few lines of code. Our experiments show that CoCoNet significantly outperforms state-of-the-art distributed machine learning implementations. @InProceedings{ASPLOS22p402, author = {Abhinav Jangda and Jun Huang and Guodong Liu and Amir Hossein Nodehi Sabet and Saeed Maleki and Youshan Miao and Madanlal Musuvathi and Todd Mytkowicz and Olli Saarikivi}, title = {Breaking the Computation and Communication Abstraction Barrier in Distributed Machine Learning Workloads}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {402--416}, doi = {10.1145/3503222.3507778}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional
	Nadeem, Hassan	ASPLOS '22: "Adelie: Continuous Address ..." Adelie: Continuous Address Space Layout Re-randomization for Linux Drivers Ruslan Nikolaev, Hassan Nadeem, Cathlyn Stone, and Binoy Ravindran (Pennsylvania State University, USA; Virginia Tech, USA) While address space layout randomization (ASLR) has been extensively studied for user-space programs, the corresponding OS kernel's KASLR support remains very limited, making the kernel vulnerable to just-in-time (JIT) return-oriented programming (ROP) attacks. Furthermore, commodity OSs such as Linux restrict their KASLR range to 32 bits due to architectural constraints (e.g., x86-64 only supports 32-bit immediate operands for most instructions), which makes them vulnerable to even unsophisticated brute-force ROP attacks due to low entropy. Most in-kernel pointers remain static, exacerbating the problem when pointers are leaked. Adelie, our kernel defense mechanism, overcomes KASLR limitations, increases KASLR entropy, and makes successful ROP attacks on the Linux kernel much harder to achieve. First, Adelie enables the position-independent code (PIC) model so that the kernel and its modules can be placed anywhere in the 64-bit virtual address space, at any distance apart from each other. Second, Adelie implements stack re-randomization and address encryption on modules. Finally, Adelie enables efficient continuous KASLR for modules by using the PIC model to make it (almost) impossible to inject ROP gadgets through these modules regardless of gadget's origin. Since device drivers (typically compiled as modules) are often developed by third parties and are typically less tested than core OS parts, they are also often more vulnerable. By fully re-randomizing device drivers, the last two contributions together prevent most JIT ROP attacks since vulnerable modules are very likely to be a starting point of an attack. Furthermore, some OS instances in virtualized environments are specifically designated to run device drivers, where drivers are the primary target of JIT ROP attacks. Using a GCC plugin that we developed, we automatically modify different kinds of kernel modules. Since the prior art tackles only user-space programs, we solve many challenges unique to the kernel code. Our evaluation shows high efficiency of Adelie's approach: the overhead of the PIC model is completely negligible and re-randomization cost remains reasonable for typical use cases. @InProceedings{ASPLOS22p483, author = {Ruslan Nikolaev and Hassan Nadeem and Cathlyn Stone and Binoy Ravindran}, title = {Adelie: Continuous Address Space Layout Re-randomization for Linux Drivers}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {483--498}, doi = {10.1145/3503222.3507779}, year = {2022}, } Publisher's Version Published Artifact Info Artifacts Available Artifacts Functional Results Reproduced
	Nair, Prashant J.	ASPLOS '22: "Randomized Row-Swap: Mitigating ..." Randomized Row-Swap: Mitigating Row Hammer by Breaking Spatial Correlation between Aggressor and Victim Rows Gururaj Saileshwar, Bolin Wang, Moinuddin Qureshi, and Prashant J. Nair (Georgia Institute of Technology, USA; University of British Columbia, Canada) Row Hammer is a fault-injection attack in which rapid activations to a single DRAM row causes bit-flips in nearby rows. Several recent defenses propose tracking aggressor-rows and applying mitigating action on neighboring victim rows by refreshing them. However, all such proposals using victim-focused mitigation preserve the spatial connection between victim and aggressor rows. Therefore, these proposals are susceptible to access patterns causing bit-flips in rows beyond the immediate neighbor. For example, the Half-Double attack causes bit-flips in the presence of victim-focused mitigation. We propose Randomized Row-Swap (RRS), a novel mitigation action that breaks the spatial connection between the aggressor and victim DRAM rows. This enables RRS to provide robust defense against even complex Row Hammer access patterns. RRS is an aggressor-focused mitigation that periodically swaps aggressor-rows with other randomly selected rows in memory. This limits the possible damage in any one locality within the DRAM memory. While RRS can be used with any tracking mechanism, we implement it with a Misra-Gries tracker and target a Row Hammer Threshold of 4.8K activations (similar to the state-of-the-art attacks). Our evaluations show that RRS has negligible slowdown (0.4% on average) and provides strong security guarantees for avoiding Row Hammer bit flips even under several years of continuous attack. @InProceedings{ASPLOS22p1056, author = {Gururaj Saileshwar and Bolin Wang and Moinuddin Qureshi and Prashant J. Nair}, title = {Randomized Row-Swap: Mitigating Row Hammer by Breaking Spatial Correlation between Aggressor and Victim Rows}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1056--1069}, doi = {10.1145/3503222.3507716}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Narayanan, Iyswarya	ASPLOS '22: "IOCost: Block IO Control for ..." IOCost: Block IO Control for Containers in Datacenters Tejun Heo, Dan Schatzberg, Andrew Newell, Song Liu, Saravanan Dhakshinamurthy, Iyswarya Narayanan, Josef Bacik, Chris Mason, Chunqiang Tang, and Dimitrios Skarlatos (Meta, USA; Carnegie Mellon University, USA) Resource isolation is a fundamental requirement in datacenter environments. However, our production experience in Meta’s large-scale datacenters shows that existing IO control mechanisms for block storage are inadequate in containerized environments. IO control needs to provide proportional resources to containers while taking into account the hardware heterogeneity of storage devices and the idiosyncrasies of the workloads deployed in datacenters. The speed of modern SSDs requires IO control to execute with low-overheads. Furthermore, IO control should strive for work conservation, take into account the interactions with the memory management subsystem, and avoid priority inversions that lead to isolation failures. To address these challenges, this paper presents IOCost, an IO control solution that is designed for containerized environments and provides scalable, work-conserving, and low-overhead IO control for heterogeneous storage devices and diverse workloads in datacenters. IOCost performs offline profiling to build a device model and uses it to estimate device occupancy of each IO request. To minimize runtime overhead, it separates IO control into a fast per-IO issue path and a slower periodic planning path. A novel work-conserving budget donation algorithm enables containers to dynamically share unused budget. We have deployed IOCost across the entirety of Meta’s datacenters comprised of millions of ma- chines, upstreamed IOCost to the Linux kernel, and open-sourced our device-profiling tools. IOCost has been running in production for two years, providing IO control for Meta’s fleet. We describe the design of IOCost and share our experience deploying it at scale. @InProceedings{ASPLOS22p595, author = {Tejun Heo and Dan Schatzberg and Andrew Newell and Song Liu and Saravanan Dhakshinamurthy and Iyswarya Narayanan and Josef Bacik and Chris Mason and Chunqiang Tang and Dimitrios Skarlatos}, title = {IOCost: Block IO Control for Containers in Datacenters}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {595--608}, doi = {10.1145/3503222.3507727}, year = {2022}, } Publisher's Version
	Nelson, Aaron	ASPLOS '22: "CARAT CAKE: Replacing Paging ..." CARAT CAKE: Replacing Paging via Compiler/Kernel Cooperation Brian Suchy, Souradip Ghosh, Drew Kersnar, Siyuan Chai, Zhen Huang, Aaron Nelson, Michael Cuevas, Alex Bernat, Gaurav Chaudhary, Nikos Hardavellas, Simone Campanoni, and Peter Dinda (Northwestern University, USA) Virtual memory, specifically paging, is undergoing significant innovation due to being challenged by new demands from modern workloads. Recent work has demonstrated an alternative software only design that can result in simplified hardware requirements, even supporting purely physical addressing. While we have made the case for this Compiler- And Runtime-based Address Translation (CARAT) concept, its evaluation was based on a user-level prototype. We now report on incorporating CARAT into a kernel, forming Compiler- And Runtime-based Address Translation for CollAborative Kernel Environments (CARAT CAKE). In our implementation, a Linux-compatible x64 process abstraction can be based either on CARAT CAKE, or on a sophisticated paging implementation. Implementing CARAT CAKE involves kernel changes and compiler optimizations/transformations that must work on all code in the system, including kernel code. We evaluate CARAT CAKE in comparison with paging and find that CARAT CAKE is able to achieve the functionality of paging (protection, mapping, and movement properties) with minimal overhead. In turn, CARAT CAKE allows significant new benefits for systems including energy savings, larger L1 caches, and arbitrary granularity memory management. @InProceedings{ASPLOS22p98, author = {Brian Suchy and Souradip Ghosh and Drew Kersnar and Siyuan Chai and Zhen Huang and Aaron Nelson and Michael Cuevas and Alex Bernat and Gaurav Chaudhary and Nikos Hardavellas and Simone Campanoni and Peter Dinda}, title = {CARAT CAKE: Replacing Paging via Compiler/Kernel Cooperation}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {98--114}, doi = {10.1145/3503222.3507771}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available
	Newell, Andrew	ASPLOS '22: "IOCost: Block IO Control for ..." IOCost: Block IO Control for Containers in Datacenters Tejun Heo, Dan Schatzberg, Andrew Newell, Song Liu, Saravanan Dhakshinamurthy, Iyswarya Narayanan, Josef Bacik, Chris Mason, Chunqiang Tang, and Dimitrios Skarlatos (Meta, USA; Carnegie Mellon University, USA) Resource isolation is a fundamental requirement in datacenter environments. However, our production experience in Meta’s large-scale datacenters shows that existing IO control mechanisms for block storage are inadequate in containerized environments. IO control needs to provide proportional resources to containers while taking into account the hardware heterogeneity of storage devices and the idiosyncrasies of the workloads deployed in datacenters. The speed of modern SSDs requires IO control to execute with low-overheads. Furthermore, IO control should strive for work conservation, take into account the interactions with the memory management subsystem, and avoid priority inversions that lead to isolation failures. To address these challenges, this paper presents IOCost, an IO control solution that is designed for containerized environments and provides scalable, work-conserving, and low-overhead IO control for heterogeneous storage devices and diverse workloads in datacenters. IOCost performs offline profiling to build a device model and uses it to estimate device occupancy of each IO request. To minimize runtime overhead, it separates IO control into a fast per-IO issue path and a slower periodic planning path. A novel work-conserving budget donation algorithm enables containers to dynamically share unused budget. We have deployed IOCost across the entirety of Meta’s datacenters comprised of millions of ma- chines, upstreamed IOCost to the Linux kernel, and open-sourced our device-profiling tools. IOCost has been running in production for two years, providing IO control for Meta’s fleet. We describe the design of IOCost and share our experience deploying it at scale. @InProceedings{ASPLOS22p595, author = {Tejun Heo and Dan Schatzberg and Andrew Newell and Song Liu and Saravanan Dhakshinamurthy and Iyswarya Narayanan and Josef Bacik and Chris Mason and Chunqiang Tang and Dimitrios Skarlatos}, title = {IOCost: Block IO Control for Containers in Datacenters}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {595--608}, doi = {10.1145/3503222.3507727}, year = {2022}, } Publisher's Version
	Nikolaev, Ruslan	ASPLOS '22: "Adelie: Continuous Address ..." Adelie: Continuous Address Space Layout Re-randomization for Linux Drivers Ruslan Nikolaev, Hassan Nadeem, Cathlyn Stone, and Binoy Ravindran (Pennsylvania State University, USA; Virginia Tech, USA) While address space layout randomization (ASLR) has been extensively studied for user-space programs, the corresponding OS kernel's KASLR support remains very limited, making the kernel vulnerable to just-in-time (JIT) return-oriented programming (ROP) attacks. Furthermore, commodity OSs such as Linux restrict their KASLR range to 32 bits due to architectural constraints (e.g., x86-64 only supports 32-bit immediate operands for most instructions), which makes them vulnerable to even unsophisticated brute-force ROP attacks due to low entropy. Most in-kernel pointers remain static, exacerbating the problem when pointers are leaked. Adelie, our kernel defense mechanism, overcomes KASLR limitations, increases KASLR entropy, and makes successful ROP attacks on the Linux kernel much harder to achieve. First, Adelie enables the position-independent code (PIC) model so that the kernel and its modules can be placed anywhere in the 64-bit virtual address space, at any distance apart from each other. Second, Adelie implements stack re-randomization and address encryption on modules. Finally, Adelie enables efficient continuous KASLR for modules by using the PIC model to make it (almost) impossible to inject ROP gadgets through these modules regardless of gadget's origin. Since device drivers (typically compiled as modules) are often developed by third parties and are typically less tested than core OS parts, they are also often more vulnerable. By fully re-randomizing device drivers, the last two contributions together prevent most JIT ROP attacks since vulnerable modules are very likely to be a starting point of an attack. Furthermore, some OS instances in virtualized environments are specifically designated to run device drivers, where drivers are the primary target of JIT ROP attacks. Using a GCC plugin that we developed, we automatically modify different kinds of kernel modules. Since the prior art tackles only user-space programs, we solve many challenges unique to the kernel code. Our evaluation shows high efficiency of Adelie's approach: the overhead of the PIC model is completely negligible and re-randomization cost remains reasonable for typical use cases. @InProceedings{ASPLOS22p483, author = {Ruslan Nikolaev and Hassan Nadeem and Cathlyn Stone and Binoy Ravindran}, title = {Adelie: Continuous Address Space Layout Re-randomization for Linux Drivers}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {483--498}, doi = {10.1145/3503222.3507779}, year = {2022}, } Publisher's Version Published Artifact Info Artifacts Available Artifacts Functional Results Reproduced
	Novaković, Stanko	ASPLOS '22: "Memory-Harvesting VMs in Cloud ..." Memory-Harvesting VMs in Cloud Platforms Alexander Fuerst, Stanko Novaković, Íñigo Goiri, Gohar Irfan Chaudhry, Prateek Sharma, Kapil Arya, Kevin Broas, Eugene Bak, Mehmet Iyigun, and Ricardo Bianchini (Indiana University, USA; Microsoft Research, USA; Microsoft Azure, USA) loud platforms monetize their spare capacity by renting “Spot” virtual machines (VMs) that can be evicted in favor of higher-priority VMs. Recent work has shown that resource-harvesting VMs are more effective at exploiting spare capacity than Spot VMs, while also reducing the number of evictions. However, the prior work focused on harvesting CPU cores while keeping memory size fixed. This wastes a substantial monetization opportunity and may even limit the ability of harvesting VMs to leverage spare cores. Thus, in this paper, we explore memory harvesting and its challenges in real cloud platforms, namely its impact on VM creation time, NUMA spanning, and page fragmentation. We start by characterizing the amount and dynamics of the spare memory in Azure. We then design and implement memory-harvesting VMs (MHVMs), introducing new techniques for memory buffering, batching, and pre-reclamation. To demonstrate the use of MHVMs, we also extend a popular cluster scheduling framework (Hadoop) and a FaaS platform to adapt to them. Our main results show that (1) there is plenty of scope for memory harvesting in real platforms; (2) MHVMs are effective at mitigating the negative impacts of harvesting; and (3) our extensions of Hadoop and FaaS successfully hide the MHVMs’ varying memory size from the users’ data-processing jobs and functions. We conclude that memory harvesting has great potential for practical deployment and users can save up to 93% of their costs when running workloads on MHVMs. @InProceedings{ASPLOS22p583, author = {Alexander Fuerst and Stanko Novaković and Íñigo Goiri and Gohar Irfan Chaudhry and Prateek Sharma and Kapil Arya and Kevin Broas and Eugene Bak and Mehmet Iyigun and Ricardo Bianchini}, title = {Memory-Harvesting VMs in Cloud Platforms}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {583--594}, doi = {10.1145/3503222.3507725}, year = {2022}, } Publisher's Version
	Nowatzki, Tony	ASPLOS '22: "TaskStream: Accelerating Task-Parallel ..." TaskStream: Accelerating Task-Parallel Workloads by Recovering Program Structure Vidushi Dadu and Tony Nowatzki (University of California at Los Angeles, USA) Reconfigurable accelerators, like CGRAs and dataflow architectures, have come to prominence for addressing data-processing problems. However, they are largely limited to workloads with regular parallelism, precluding their applicability to prevalent task-parallel workloads. Reconfigurable architectures and task parallelism seem to be at odds, as the former requires repetitive and simple program structure, and the latter breaks program structure to create small, individually scheduled program units. Our insight is that if tasks and their potential for communication structure are first-class primitives in the hardware, it is possible to recover program structure with extremely low overhead. We propose a task execution model for accelerators called TaskStream, which annotates task dependences with information sufficient to recover inter-task structure. TaskStream enables work-aware load balancing, recovery of pipelined inter-task dependences, and recovery of inter-task read sharing through multicasting. We apply TaskStream to a reconfigurable dataflow architecture, creating a seamless hierarchical dataflow model for task-parallel workloads. We compare our accelerator, Delta, with an equivalent static-parallel design. Overall, we find that our execution model can improve performance by 2.2× with only 3.6% area overhead, while alleviating the programming burden of managing task distribution. @InProceedings{ASPLOS22p1, author = {Vidushi Dadu and Tony Nowatzki}, title = {TaskStream: Accelerating Task-Parallel Workloads by Recovering Program Structure}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1--13}, doi = {10.1145/3503222.3507706}, year = {2022}, } Publisher's Version
	Oest, Adam	ASPLOS '22: "ViK: Practical Mitigation ..." ViK: Practical Mitigation of Temporal Memory Safety Violations through Object ID Inspection Haehyun Cho, Jinbum Park, Adam Oest, Tiffany Bao, Ruoyu Wang, Yan Shoshitaishvili, Adam Doupé, and Gail-Joon Ahn (Soongsil University, South Korea; Samsung Research, South Korea; PayPal, USA; Arizona State University, USA; Samsung Research, USA) Temporal memory safety violations, such as use-after-free (UAF) vulnerabilities, are a critical security issue for software written in memory-unsafe languages such as C and C++. In this paper, we introduce ViK, a novel, lightweight, and widely applicable runtime defense that can protect both operating system (OS) kernels and user-space applications against temporal memory safety violations. ViK performs object ID inspection, where it assigns a random identifier to every allocated object and stores the identifier in the unused bits of the corresponding pointer. When the pointer is used, ViK inspects the value of a pointer before dereferencing, ensuring that the pointer still references the original object. To the best of our knowledge, this is the first mitigation against temporal memory safety violations that scales to OS kernels. We evaluated the software prototype of ViK on Android and Linux kernels and observed runtime overhead of around 20%. Also, we evaluated a hardware-assisted prototype of ViK on Android kernel, where the runtime overhead was as low as 2%. @InProceedings{ASPLOS22p271, author = {Haehyun Cho and Jinbum Park and Adam Oest and Tiffany Bao and Ruoyu Wang and Yan Shoshitaishvili and Adam Doupé and Gail-Joon Ahn}, title = {ViK: Practical Mitigation of Temporal Memory Safety Violations through Object ID Inspection}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {271--284}, doi = {10.1145/3503222.3507780}, year = {2022}, } Publisher's Version Artifacts Functional
	Oleksenko, Oleksii	ASPLOS '22: "Revizor: Testing Black-Box ..." Revizor: Testing Black-Box CPUs against Speculation Contracts Oleksii Oleksenko, Christof Fetzer, Boris Köpf, and Mark Silberstein (TU Dresden, Germany; Microsoft Research, UK; Technion, Israel) Speculative vulnerabilities such as Spectre and Meltdown expose speculative execution state that can be exploited to leak information across security domains via side-channels. Such vulnerabilities often stay undetected for a long time as we lack the tools for systematic testing of CPUs to find them. In this paper, we propose an approach to automatically detect microarchitectural information leakage in commercial black-box CPUs. We build on speculation contracts, which we employ to specify the permitted side effects of program execution on the CPU's microarchitectural state. We propose a Model-based Relational Testing (MRT) technique to empirically assess the CPU compliance with these specifications. We implement MRT in a testing framework called Revizor, and showcase its effectiveness on real Intel x86 CPUs. Revizor automatically detects violations of a rich set of contracts, or indicates their absence. A highlight of our findings is that Revizor managed to automatically surface Spectre, MDS, and LVI, as well as several previously unknown variants. @InProceedings{ASPLOS22p226, author = {Oleksii Oleksenko and Christof Fetzer and Boris Köpf and Mark Silberstein}, title = {Revizor: Testing Black-Box CPUs against Speculation Contracts}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {226--239}, doi = {10.1145/3503222.3507729}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Olgun, Ataberk	ASPLOS '22: "GenStore: A High-Performance ..." GenStore: A High-Performance In-Storage Processing System for Genome Sequence Analysis Nika Mansouri Ghiasi, Jisung Park, Harun Mustafa, Jeremie Kim, Ataberk Olgun, Arvid Gollwitzer, Damla Senol Cali, Can Firtina, Haiyu Mao, Nour Almadhoun Alserr, Rachata Ausavarungnirun, Nandita Vijaykumar, Mohammed Alser, and Onur Mutlu (ETH Zurich, Switzerland; Bionano Genomics, USA; King Mongkut's University of Technology North Bangkok, Thailand; University of Toronto, Canada) Read mapping is a fundamental step in many genomics applications. It is used to identify potential matches and differences between fragments (called reads) of a sequenced genome and an already known genome (called a reference genome). Read mapping is costly because it needs to perform approximate string matching (ASM) on large amounts of data. To address the computational challenges in genome analysis, many prior works propose various approaches such as accurate filters that select the reads within a dataset of genomic reads (called a read set) that must undergo expensive computation, efficient heuristics, and hardware acceleration. While effective at reducing the amount of expensive computation, all such approaches still require the costly movement of a large amount of data from storage to the rest of the system, which can significantly lower the end-to-end performance of read mapping in conventional and emerging genomics systems. We propose GenStore, the first in-storage processing system designed for genome sequence analysis that greatly reduces both data movement and computational overheads of genome sequence analysis by exploiting low-cost and accurate in-storage filters. GenStore leverages hardware/software co-design to address the challenges of in-storage processing, supporting reads with 1) different properties such as read lengths and error rates, which highly depend on the sequencing technology, and 2) different degrees of genetic variation compared to the reference genome, which highly depends on the genomes that are being compared. Through rigorous analysis of read mapping processes of reads with different properties and degrees of genetic variation, we meticulously design low-cost hardware accelerators and data/computation flows inside a NAND flash-based solid-state drive (SSD). Our evaluation using a wide range of real genomic datasets shows that GenStore, when implemented in three modern NAND flash-based SSDs, significantly improves the read mapping performance of state-of-the-art software (hardware) baselines by 2.07-6.05× (1.52-3.32×) for read sets with high similarity to the reference genome and 1.45-33.63× (2.70-19.2×) for read sets with low similarity to the reference genome. @InProceedings{ASPLOS22p635, author = {Nika Mansouri Ghiasi and Jisung Park and Harun Mustafa and Jeremie Kim and Ataberk Olgun and Arvid Gollwitzer and Damla Senol Cali and Can Firtina and Haiyu Mao and Nour Almadhoun Alserr and Rachata Ausavarungnirun and Nandita Vijaykumar and Mohammed Alser and Onur Mutlu}, title = {GenStore: A High-Performance In-Storage Processing System for Genome Sequence Analysis}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {635--654}, doi = {10.1145/3503222.3507702}, year = {2022}, } Publisher's Version
	Olivier, Pierre	ASPLOS '22: "FlexOS: Towards Flexible OS ..." FlexOS: Towards Flexible OS Isolation Hugo Lefeuvre, Vlad-Andrei Bădoiu, Alexander Jung, Stefan Lucian Teodorescu, Sebastian Rauch, Felipe Huici, Costin Raiciu, and Pierre Olivier (University of Manchester, UK; University Politehnica of Bucharest, Romania; Lancaster University, UK; unikraft.io, Germany; KIT, Germany; NEC Laboratories, Germany; Correct Networks, Romania) At design time, modern operating systems are locked in a specific safety and isolation strategy that mixes one or more hardware/software protection mechanisms (e.g. user/kernel separation); revisiting these choices after deployment requires a major refactoring effort. This rigid approach shows its limits given the wide variety of modern applications' safety/performance requirements, when new hardware isolation mechanisms are rolled out, or when existing ones break. We present FlexOS, a novel OS allowing users to easily specialize the safety and isolation strategy of an OS at compilation/deployment time instead of design time. This modular LibOS is composed of fine-grained components that can be isolated via a range of hardware protection mechanisms with various data sharing strategies and additional software hardening. The OS ships with an exploration technique helping the user navigate the vast safety/performance design space it unlocks. We implement a prototype of the system and demonstrate, for several applications (Redis/Nginx/SQLite), FlexOS' vast configuration space as well as the efficiency of the exploration technique: we evaluate 80 FlexOS configurations for Redis and show how that space can be probabilistically subset to the 5 safest ones under a given performance budget. We also show that, under equivalent configurations, FlexOS performs similarly or better than existing solutions which use fixed safety configurations. @InProceedings{ASPLOS22p467, author = {Hugo Lefeuvre and Vlad-Andrei Bădoiu and Alexander Jung and Stefan Lucian Teodorescu and Sebastian Rauch and Felipe Huici and Costin Raiciu and Pierre Olivier}, title = {FlexOS: Towards Flexible OS Isolation}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {467--482}, doi = {10.1145/3503222.3507759}, year = {2022}, } Publisher's Version Published Artifact Info Artifacts Available Artifacts Functional Results Reproduced
	Olivieri, Mauro	ASPLOS '22: "BiSon-e: A Lightweight and ..." BiSon-e: A Lightweight and High-Performance Accelerator for Narrow Integer Linear Algebra Computing on the Edge Enrico Reggiani, Cristóbal Ramírez Lazo, Roger Figueras Bagué, Adrián Cristal, Mauro Olivieri, and Osman Sabri Unsal (Polytechnic University of Catalonia, Spain; Barcelona Supercomputing Center, Spain; Sapienza University of Rome, Italy) Linear algebra computational kernels based on byte and sub-byte integer data formats are at the base of many classes of applications, ranging from Deep Learning to Pattern Matching. Porting the computation of these applications from cloud to edge and mobile devices would enable significant improvements in terms of security, safety, and energy efficiency. However, despite their low memory and energy demands, their intrinsically high computational intensity makes the execution of these workloads challenging on highly resource-constrained devices. In this paper, we present BiSon-e, a novel RISC-V based architecture that accelerates linear algebra kernels based on narrow integer computations on edge processors by performing Single Instruction Multiple Data (SIMD) operations on off-the-shelf scalar Functional Units (FUs). Our novel architecture is built upon the binary segmentation technique, which allows to significantly reduce the memory footprint and the arithmetic intensity of linear algebra kernels requiring narrow data sizes. We integrate BiSon-e into a complete System-on-Chip (SoC) based on RISC-V, synthesized and Place&Routed in 65nm and 22nm technologies, introducing a negligible 0.07% area overhead with respect to the baseline architecture. Our experimental evaluation shows that, when computing the Convolution and Fully-Connected layers of the AlexNet and VGG-16 Convolutional Neural Networks (CNNs) with 8-, 4-, and 2-bit, our solution gains up to 5.6×, 13.9× and 24× in execution time compared to the scalar implementation of a single RISC-V core, and improves the energy efficiency of string matching tasks by 5× when compared to a RISC-V-based Vector Processing Unit (VPU). @InProceedings{ASPLOS22p56, author = {Enrico Reggiani and Cristóbal Ramírez Lazo and Roger Figueras Bagué and Adrián Cristal and Mauro Olivieri and Osman Sabri Unsal}, title = {BiSon-e: A Lightweight and High-Performance Accelerator for Narrow Integer Linear Algebra Computing on the Edge}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {56--69}, doi = {10.1145/3503222.3507746}, year = {2022}, } Publisher's Version
	Olukotun, Kunle	ASPLOS '22: "Taurus: A Data Plane Architecture ..." Taurus: A Data Plane Architecture for Per-Packet ML Tushar Swamy, Alexander Rucker, Muhammad Shahbaz, Ishan Gaur, and Kunle Olukotun (Stanford University, USA; Purdue University, USA) Emerging applications---cloud computing, the internet of things, and augmented/virtual reality---demand responsive, secure, and scalable datacenter networks. These networks currently implement simple, per-packet, data-plane heuristics (e.g., ECMP and sketches) under a slow, millisecond-latency control plane that runs data-driven performance and security policies. However, to meet applications' service-level objectives (SLOs) in a modern data center, networks must bridge the gap between line-rate, per-packet execution and complex decision making. In this work, we present the design and implementation of Taurus, a data plane for line-rate inference. Taurus adds custom hardware based on a flexible, parallel-patterns (MapReduce) abstraction to programmable network devices, such as switches and NICs; this new hardware uses pipelined SIMD parallelism to enable per-packet MapReduce operations (e.g., inference). Our evaluation of a Taurus switch ASIC---supporting several real-world models---shows that Taurus operates orders of magnitude faster than a server-based control plane while increasing area by 3.8% and latency for line-rate ML models by up to 221 ns. Furthermore, our Taurus FPGA prototype achieves full model accuracy and detects two orders of magnitude more events than a state-of-the-art control-plane anomaly-detection system. @InProceedings{ASPLOS22p1099, author = {Tushar Swamy and Alexander Rucker and Muhammad Shahbaz and Ishan Gaur and Kunle Olukotun}, title = {Taurus: A Data Plane Architecture for Per-Packet ML}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1099--1114}, doi = {10.1145/3503222.3507726}, year = {2022}, } Publisher's Version Published Artifact Info Artifacts Available
	Pandey, Shweta	ASPLOS '22: "GPM: Leveraging Persistent ..." GPM: Leveraging Persistent Memory from a GPU Shweta Pandey, Aditya K Kamath, and Arkaprava Basu (IISc Bangalore, India) The GPU is a key computing platform for many application domains. While the new non-volatile memory technology has brought the promise of byte-addressable persistence (a.k.a., persistent memory, or PM) to CPU applications, the same, unfortunately, is beyond the reach of GPU programs. We take three key steps toward enabling GPU programs to access PM directly. First, enable direct access to PM from within a GPU kernel without needing to modify the hardware. Next, we demonstrate three classes of GPU-accelerated applications that benefit from PM. In the process, we create a workload suite with nine such applications. We then create a GPU library, written in CUDA, to support logging, checkpointing, and primitives for native persistence for programmers to easily leverage PM. @InProceedings{ASPLOS22p142, author = {Shweta Pandey and Aditya K Kamath and Arkaprava Basu}, title = {GPM: Leveraging Persistent Memory from a GPU}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {142--156}, doi = {10.1145/3503222.3507758}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Park, Chang Hyun	ASPLOS '22: "Every Walk’s a Hit: Making ..." Every Walk’s a Hit: Making Page Walks Single-Access Cache Hits Chang Hyun Park, Ilias Vougioukas, Andreas Sandberg, and David Black-Schaffer (Uppsala University, Sweden; Arm Research, UK) As memory capacity has outstripped TLB coverage, large data applications suffer from frequent page table walks. We investigate two complementary techniques for addressing this cost: reducing the number of accesses required and reducing the latency of each access. The first approach is accomplished by opportunistically "flattening" the page table: merging two levels of traditional 4 KB page table nodes into a single 2 MB node, thereby reducing the table's depth and the number of indirections required to traverse it. The second is accomplished by biasing the cache replacement algorithm to keep page table entries during periods of high TLB miss rates, as these periods also see high data miss rates and are therefore more likely to benefit from having the smaller page table in the cache than to suffer from increased data cache misses. We evaluate these approaches for both native and virtualized systems and across a range of realistic memory fragmentation scenarios, describe the limited changes needed in our kernel implementation and hardware design, identify and address challenges related to self-referencing page tables and kernel memory allocation, and compare results across server and mobile systems using both academic and industrial simulators for robustness. We find that flattening does reduce the number of accesses required on a page walk (to 1.0), but its performance impact (+2.3%) is small due to Page Walker Caches (already 1.5 accesses). Prioritizing caching has a larger effect (+6.8%), and the combination improves performance by +9.2%. Flattening is more effective on virtualized systems (4.4 to 2.8 accesses, +7.1% performance), due to 2D page walks. By combining the two techniques we demonstrate a state-of-the-art +14.0% performance gain and -8.7% dynamic cache energy and -4.7% dynamic DRAM energy for virtualized execution with very simple hardware and software changes. @InProceedings{ASPLOS22p128, author = {Chang Hyun Park and Ilias Vougioukas and Andreas Sandberg and David Black-Schaffer}, title = {Every Walk’s a Hit: Making Page Walks Single-Access Cache Hits}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {128--141}, doi = {10.1145/3503222.3507718}, year = {2022}, } Publisher's Version
	Park, Heejin	ASPLOS '22: "GPUReplay: A 50-KB GPU Stack ..." GPUReplay: A 50-KB GPU Stack for Client ML Heejin Park and Felix Xiaozhu Lin (Purdue University, USA; University of Virginia, USA) GPUReplay (GR) is a novel way for deploying GPU-accelerated computation on mobile and embedded devices. It addresses high complexity of a modern GPU stack for deployment ease and security. The idea is to record GPU executions on the full GPU stack ahead of time and replay the executions on new input at run time. We address key challenges towards making GR feasible, sound, and practical to use. The resultant replayer is a drop-in replacement of the original GPU stack. It is tiny (50 KB of executable), robust (replaying long executions without divergence), portable (running in a commodity OS, in TEE, and baremetal), and quick to launch (speeding up startup by up to two orders of magnitude). We show that GPUReplay works with a variety of integrated GPU hardware, GPU APIs, ML frameworks, and 33 neural network (NN) implementations for inference or training. The code is available at https://github.com/bakhi/GPUReplay. @InProceedings{ASPLOS22p157, author = {Heejin Park and Felix Xiaozhu Lin}, title = {GPUReplay: A 50-KB GPU Stack for Client ML}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {157--170}, doi = {10.1145/3503222.3507754}, year = {2022}, } Publisher's Version
	Park, Jinbum	ASPLOS '22: "ViK: Practical Mitigation ..." ViK: Practical Mitigation of Temporal Memory Safety Violations through Object ID Inspection Haehyun Cho, Jinbum Park, Adam Oest, Tiffany Bao, Ruoyu Wang, Yan Shoshitaishvili, Adam Doupé, and Gail-Joon Ahn (Soongsil University, South Korea; Samsung Research, South Korea; PayPal, USA; Arizona State University, USA; Samsung Research, USA) Temporal memory safety violations, such as use-after-free (UAF) vulnerabilities, are a critical security issue for software written in memory-unsafe languages such as C and C++. In this paper, we introduce ViK, a novel, lightweight, and widely applicable runtime defense that can protect both operating system (OS) kernels and user-space applications against temporal memory safety violations. ViK performs object ID inspection, where it assigns a random identifier to every allocated object and stores the identifier in the unused bits of the corresponding pointer. When the pointer is used, ViK inspects the value of a pointer before dereferencing, ensuring that the pointer still references the original object. To the best of our knowledge, this is the first mitigation against temporal memory safety violations that scales to OS kernels. We evaluated the software prototype of ViK on Android and Linux kernels and observed runtime overhead of around 20%. Also, we evaluated a hardware-assisted prototype of ViK on Android kernel, where the runtime overhead was as low as 2%. @InProceedings{ASPLOS22p271, author = {Haehyun Cho and Jinbum Park and Adam Oest and Tiffany Bao and Ruoyu Wang and Yan Shoshitaishvili and Adam Doupé and Gail-Joon Ahn}, title = {ViK: Practical Mitigation of Temporal Memory Safety Violations through Object ID Inspection}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {271--284}, doi = {10.1145/3503222.3507780}, year = {2022}, } Publisher's Version Artifacts Functional
	Park, Jisung	ASPLOS '22: "GenStore: A High-Performance ..." GenStore: A High-Performance In-Storage Processing System for Genome Sequence Analysis Nika Mansouri Ghiasi, Jisung Park, Harun Mustafa, Jeremie Kim, Ataberk Olgun, Arvid Gollwitzer, Damla Senol Cali, Can Firtina, Haiyu Mao, Nour Almadhoun Alserr, Rachata Ausavarungnirun, Nandita Vijaykumar, Mohammed Alser, and Onur Mutlu (ETH Zurich, Switzerland; Bionano Genomics, USA; King Mongkut's University of Technology North Bangkok, Thailand; University of Toronto, Canada) Read mapping is a fundamental step in many genomics applications. It is used to identify potential matches and differences between fragments (called reads) of a sequenced genome and an already known genome (called a reference genome). Read mapping is costly because it needs to perform approximate string matching (ASM) on large amounts of data. To address the computational challenges in genome analysis, many prior works propose various approaches such as accurate filters that select the reads within a dataset of genomic reads (called a read set) that must undergo expensive computation, efficient heuristics, and hardware acceleration. While effective at reducing the amount of expensive computation, all such approaches still require the costly movement of a large amount of data from storage to the rest of the system, which can significantly lower the end-to-end performance of read mapping in conventional and emerging genomics systems. We propose GenStore, the first in-storage processing system designed for genome sequence analysis that greatly reduces both data movement and computational overheads of genome sequence analysis by exploiting low-cost and accurate in-storage filters. GenStore leverages hardware/software co-design to address the challenges of in-storage processing, supporting reads with 1) different properties such as read lengths and error rates, which highly depend on the sequencing technology, and 2) different degrees of genetic variation compared to the reference genome, which highly depends on the genomes that are being compared. Through rigorous analysis of read mapping processes of reads with different properties and degrees of genetic variation, we meticulously design low-cost hardware accelerators and data/computation flows inside a NAND flash-based solid-state drive (SSD). Our evaluation using a wide range of real genomic datasets shows that GenStore, when implemented in three modern NAND flash-based SSDs, significantly improves the read mapping performance of state-of-the-art software (hardware) baselines by 2.07-6.05× (1.52-3.32×) for read sets with high similarity to the reference genome and 1.45-33.63× (2.70-19.2×) for read sets with low similarity to the reference genome. @InProceedings{ASPLOS22p635, author = {Nika Mansouri Ghiasi and Jisung Park and Harun Mustafa and Jeremie Kim and Ataberk Olgun and Arvid Gollwitzer and Damla Senol Cali and Can Firtina and Haiyu Mao and Nour Almadhoun Alserr and Rachata Ausavarungnirun and Nandita Vijaykumar and Mohammed Alser and Onur Mutlu}, title = {GenStore: A High-Performance In-Storage Processing System for Genome Sequence Analysis}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {635--654}, doi = {10.1145/3503222.3507702}, year = {2022}, } Publisher's Version
	Patel, Tirthak	ASPLOS '22: "QUEST: Systematically Approximating ..." QUEST: Systematically Approximating Quantum Circuits for Higher Output Fidelity Tirthak Patel, Ed Younis, Costin Iancu, Wibe de Jong, and Devesh Tiwari (Northeastern University, USA; Lawrence Berkeley National Laboratory, USA) We present QUEST, a procedure to systematically generate approximations for quantum circuits to reduce their CNOT gate count. Our approach employs circuit partitioning for scalability with procedures to 1) reduce circuit length using approximate synthesis, 2) improve fidelity by running circuits that represent key samples in the approximation space, and 3) reason about approximation upper bound. Our evaluation results indicate that our approach of "dissimilar" approximations provides close fidelity to the original circuit. Overall, the results indicate that QUEST can reduce CNOT gate count by 30-80% on ideal systems and decrease the impact of noise on existing and near-future quantum systems. @InProceedings{ASPLOS22p514, author = {Tirthak Patel and Ed Younis and Costin Iancu and Wibe de Jong and Devesh Tiwari}, title = {QUEST: Systematically Approximating Quantum Circuits for Higher Output Fidelity}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {514--528}, doi = {10.1145/3503222.3507739}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional ASPLOS '22: "IceBreaker: Warming Serverless ..." IceBreaker: Warming Serverless Functions Better with Heterogeneity Rohan Basu Roy, Tirthak Patel, and Devesh Tiwari (Northeastern University, USA) Serverless computing, an emerging computing model, relies on "warming up" functions prior to its anticipated execution for faster and cost-effective service to users. Unfortunately, warming up functions can be inaccurate and incur prohibitively expensive cost during the warmup period (i.e., keep-alive cost). In this paper, we introduce IceBreaker, a novel technique that reduces the service time and the "keep-alive" cost by composing a system with heterogeneous nodes (costly and cheaper). IceBreaker does so by dynamically determining the cost-effective node type to warm up a function based on the function's time-varying probability of the next invocation. By employing heterogeneity, IceBreaker allows for more number of nodes under the same cost budget and hence, keeps more number of functions warm and reduces the wait time during high load. Our real-system evaluation confirms that IceBreaker reduces the overall keep-alive cost by 45% and execution time by 27% using representative serverless applications and industry-grade workload trace. IceBreaker is the first technique to employ and leverage the idea of mixing expensive and cheaper nodes to improve both service time and keep-alive cost for serverless functions -- opening up a new research avenue of serverless computing on heterogeneous servers for researchers and practitioners. @InProceedings{ASPLOS22p753, author = {Rohan Basu Roy and Tirthak Patel and Devesh Tiwari}, title = {IceBreaker: Warming Serverless Functions Better with Heterogeneity}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {753--767}, doi = {10.1145/3503222.3507750}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Pavlogiannis, Andreas	ASPLOS '22: "A Tree Clock Data Structure ..." A Tree Clock Data Structure for Causal Orderings in Concurrent Executions Umang Mathur, Andreas Pavlogiannis, Hünkar Can Tunç, and Mahesh Viswanathan (National University of Singapore, Singapore; Aarhus University, Denmark; University of Illinois at Urbana-Champaign, USA) Dynamic techniques are a scalable and effective way to analyze concurrent programs. Instead of analyzing all behaviors of a program, these techniques detect errors by focusing on a single program execution. Often a crucial step in these techniques is to define a causal ordering between events in the execution, which is then computed using vector clocks, a simple data structure that stores logical times of threads. The two basic operations of vector clocks, namely join and copy, require Θ(k) time, where k is the number of threads. Thus they are a computational bottleneck when k is large. In this work, we introduce tree clocks, a new data structure that replaces vector clocks for computing causal orderings in program executions. Joining and copying tree clocks takes time that is roughly proportional to the number of entries being modified, and hence the two operations do not suffer the a-priori Θ(k) cost per application. We show that when used to compute the classic happens-before (HB) partial order, tree clocks are optimal, in the sense that no other data structure can lead to smaller asymptotic running time. Moreover, we demonstrate that tree clocks can be used to compute other partial orders, such as schedulable-happens-before (SHB) and the standard Mazurkiewicz (MAZ) partial order, and thus are a versatile data structure. Our experiments show that just by replacing vector clocks with tree clocks, the computation becomes from 2.02 × faster (MAZ) to 2.66 × (SHB) and 2.97 × (HB) on average per benchmark. These results illustrate that tree clocks have the potential to become a standard data structure with wide applications in concurrent analyses. @InProceedings{ASPLOS22p710, author = {Umang Mathur and Andreas Pavlogiannis and Hünkar Can Tunç and Mahesh Viswanathan}, title = {A Tree Clock Data Structure for Causal Orderings in Concurrent Executions}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {710--725}, doi = {10.1145/3503222.3507734}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Peh, Li-Shiuan	ASPLOS '22: "REVAMP: A Systematic Framework ..." REVAMP: A Systematic Framework for Heterogeneous CGRA Realization Thilini Kaushalya Bandara, Dhananjaya Wijerathne, Tulika Mitra, and Li-Shiuan Peh (National University of Singapore, Singapore) Coarse-Grained Reconfigurable Architectures (CGRAs) provide an excellent balance between performance, energy efficiency, and flexibility. However, increasingly sophisticated applications, especially on the edge devices, demand even better energy efficiency for longer battery life. Most CGRAs adhere to a canonical structure where a homogeneous set of processing elements and memories communicate through a regular interconnect due to the simplicity of the design. Unfortunately, the homogeneity leads to substantial idle resources while mapping irregular applications and creates inefficiency. We plan to mitigate the inefficiency by systematically and judiciously introducing heterogeneity in CGRAs in tandem with appropriate compiler support. We propose REVAMP, an automated design space exploration framework that helps architects uncover and add pertinent heterogeneity to a diverse range of originally homogeneous CGRAs when fed with a suite of target applications. REVAMP explores a comprehensive set of optimizations encompassing compute, network, and memory heterogeneity, thereby converting a uniform CGRA into a more irregular architecture with improved energy efficiency. As CGRAs are inherently software scheduled, any micro-architectural optimizations need to be partnered with corresponding compiler support, which is challenging with heterogeneity. The REVAMP framework extends compiler support for efficient mapping of loop kernels on the derived heterogeneous CGRA architectures. We showcase REVAMP on three state-of-the-art homogeneous CGRAs, demonstrating how REVAMP derives a heterogeneous variant of each homogeneous architecture, with its corresponding compiler optimizations. Our results show that the derived heterogeneous architectures achieve up to 52.4% power reduction, 38.1% area reduction, and 36% average energy reduction over the corresponding homogeneous versions with minimal performance impact for the selected kernel suite. @InProceedings{ASPLOS22p918, author = {Thilini Kaushalya Bandara and Dhananjaya Wijerathne and Tulika Mitra and Li-Shiuan Peh}, title = {REVAMP: A Systematic Framework for Heterogeneous CGRA Realization}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {918--932}, doi = {10.1145/3503222.3507772}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Pei, Qiangyu	ASPLOS '22: "CoolEdge: Hotspot-Relievable ..." CoolEdge: Hotspot-Relievable Warm Water Cooling for Energy-Efficient Edge Datacenters Qiangyu Pei, Shutong Chen, Qixia Zhang, Xinhui Zhu, Fangming Liu, Ziyang Jia, Yishuo Wang, and Yongjie Yuan (Huazhong University of Science and Technology, China) As the computing frontier drifts to the edge, edge datacenters play a crucial role in supporting various real-time applications. Different from cloud datacenters, the requirements of proximity to end-users, high density, and heterogeneity, present new challenges to cool the edge datacenters efficiently. Although warm water cooling has become a promising cooling technique for this infrastructure, the one-size-fits-all cooling control would lower the cooling efficiency considerably because of the severe thermal imbalance across servers, hardware, and even inside one hardware component in an edge datacenter. In this work, we propose CoolEdge, a hotspot-relievable warm water cooling system for improving the cooling efficiency and saving costs of edge datacenters. Specifically, through the elaborate design of water circulations, CoolEdge can dynamically adjust the water temperature and flow rate for each heterogeneous hardware component to eliminate the hardware-level hotspots. By redesigning cold plates, CoolEdge can quickly disperse the chip-level hotspots without manual intervention. We further quantify the power saving achieved by the warm water cooling theoretically, and propose a custom-designed cooling solution to decide an appropriate water temperature and flow rate periodically. Based on a hardware prototype and real-world traces from SURFsara, the evaluation results show that CoolEdge reduces the cooling energy by 81.81% and 71.92%, respectively, compared with conventional and state-of-the-art water cooling systems. @InProceedings{ASPLOS22p814, author = {Qiangyu Pei and Shutong Chen and Qixia Zhang and Xinhui Zhu and Fangming Liu and Ziyang Jia and Yishuo Wang and Yongjie Yuan}, title = {CoolEdge: Hotspot-Relievable Warm Water Cooling for Energy-Efficient Edge Datacenters}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {814--829}, doi = {10.1145/3503222.3507713}, year = {2022}, } Publisher's Version
	Pismenny, Boris	ASPLOS '22: "The Benefits of General-Purpose ..." The Benefits of General-Purpose On-NIC Memory Boris Pismenny, Liran Liss, Adam Morrison, and Dan Tsafrir (Technion, Israel; NVIDIA, Israel; Tel Aviv University, Israel; VMware Research, Israel) We propose to use the small, newly available on-NIC memory ("nicmem") to keep pace with the rapidly increasing performance of NICs. We motivate our proposal by accelerating two types of workload classes: NFV and key-value stores. As NFV workloads frequently operate on headers---rather than data---of incoming packets, we introduce a new packet-processing architecture that splits between the two, keeping the data on nicmem when possible and thus reducing PCIe traffic, memory bandwidth, and CPU processing time. Our approach consequently shortens NFV latency by up to 23% and increases its throughput by up to 19%. Similarly, because key-value stores commonly exhibit skewed distributions, we introduce a new network stack mechanism that lets applications keep frequently accessed items on nicmem. Our design shortens memcached latency by up to 43% and increases its throughput by up to 80%. @InProceedings{ASPLOS22p1130, author = {Boris Pismenny and Liran Liss and Adam Morrison and Dan Tsafrir}, title = {The Benefits of General-Purpose On-NIC Memory}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1130--1147}, doi = {10.1145/3503222.3507711}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Prabhu, Kartik	ASPLOS '22: "A Full-Stack Search Technique ..." A Full-Stack Search Technique for Domain Optimized Deep Learning Accelerators Dan Zhang, Safeen Huda, Ebrahim Songhori, Kartik Prabhu, Quoc Le, Anna Goldie, and Azalia Mirhoseini (Google Brain, USA; Google, USA; Stanford University, USA) The rapidly-changing deep learning landscape presents a unique opportunity for building inference accelerators optimized for specific datacenter-scale workloads. We propose Full-stack Accelerator Search Technique (FAST), a hardware accelerator search framework that defines a broad optimization environment covering key design decisions within the hardware-software stack, including hardware datapath, software scheduling, and compiler passes such as operation fusion and tensor padding. In this paper, we analyze bottlenecks in state-of-the-art vision and natural language processing (NLP) models, including EfficientNet and BERT, and use FAST to design accelerators capable of addressing these bottlenecks. FAST-generated accelerators optimized for single workloads improve Perf/TDP by 3.7× on average across all benchmarks compared to TPU-v3. A FAST-generated accelerator optimized for serving a suite of workloads improves Perf/TDP by 2.4× on average compared to TPU-v3. Our return on investment analysis shows that FAST-generated accelerators can potentially be practical for moderate-sized datacenter deployments. @InProceedings{ASPLOS22p27, author = {Dan Zhang and Safeen Huda and Ebrahim Songhori and Kartik Prabhu and Quoc Le and Anna Goldie and Azalia Mirhoseini}, title = {A Full-Stack Search Technique for Domain Optimized Deep Learning Accelerators}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {27--42}, doi = {10.1145/3503222.3507767}, year = {2022}, } Publisher's Version
	Qian, Xuehai	ASPLOS '22: "SparseCore: Stream ISA and ..." SparseCore: Stream ISA and Processor Specialization for Sparse Computation Gengyu Rao, Jingji Chen, Jason Yik, and Xuehai Qian (University of Southern California, USA) Computation on sparse data is becoming increasingly important for many applications. Recent sparse computation accelerators are designed for specific algorithm/application, making them inflexible with software optimizations. This paper proposes SparseCore, the first general-purpose processor extension for sparse computation that can flexibly accelerate complex code patterns and fast-evolving algorithms. We extend the instruction set architecture (ISA) to make stream or sparse vector first-class citizens, and develop efficient architectural components to support the stream ISA. The novel ISA extension intrinsically operates on streams, realizing both efficient data movement and computation. The simulation results show that SparseCore achieves significant speedups for sparse tensor computation and graph pattern computation. @InProceedings{ASPLOS22p186, author = {Gengyu Rao and Jingji Chen and Jason Yik and Xuehai Qian}, title = {SparseCore: Stream ISA and Processor Specialization for Sparse Computation}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {186--199}, doi = {10.1145/3503222.3507705}, year = {2022}, } Publisher's Version
	Qu, Zheng	ASPLOS '22: "DOTA: Detect and Omit Weak ..." DOTA: Detect and Omit Weak Attentions for Scalable Transformer Acceleration Zheng Qu, Liu Liu, Fengbin Tu, Zhaodong Chen, Yufei Ding, and Yuan Xie (University of California at Santa Barbara, USA) Transformer Neural Networks have demonstrated leading performance in many applications spanning over language understanding, image processing, and generative modeling. Despite the impressive performance, long-sequence Transformer processing is expensive due to quadratic computation complexity and memory consumption of self-attention. In this paper, we present DOTA, an algorithm-architecture co-design that effectively addresses the challenges of scalable Transformer inference. Based on the insight that not all connections in an attention graph are equally important, we propose to jointly optimize a lightweight Detector with the Transformer model to accurately detect and omit weak connections during runtime. Furthermore, we design a specialized system architecture for end-to-end Transformer acceleration using the proposed attention detection mechanism. Experiments on a wide range of benchmarks demonstrate the superior performance of DOTA over other solutions. In summary, DOTA achieves 152.6x and 4.5x performance speedup and orders of magnitude energy-efficiency improvements over GPU and customized hardware, respectively. @InProceedings{ASPLOS22p14, author = {Zheng Qu and Liu Liu and Fengbin Tu and Zhaodong Chen and Yufei Ding and Yuan Xie}, title = {DOTA: Detect and Omit Weak Attentions for Scalable Transformer Acceleration}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {14--26}, doi = {10.1145/3503222.3507738}, year = {2022}, } Publisher's Version
	Quinn, Andrew	ASPLOS '22: "Debugging in the Brave New ..." Debugging in the Brave New World of Reconfigurable Hardware Jiacheng Ma, Gefei Zuo, Kevin Loughlin, Haoyang Zhang, Andrew Quinn, and Baris Kasikci (University of Michigan, USA; University of California at Santa Cruz, USA) Software and hardware development cycles have traditionally been quite distinct. Software allows post-deployment patches, which leads to a rapid development cycle. In contrast, hardware bugs that are found after fabrication are extremely costly to fix (and sometimes even unfixable), so the traditional hardware development cycle involves massive investment in extensive simulation and formal verification. Reconfigurable hardware, such as a Field Programmable Gate Array (FPGA), promises to propel hardware development towards an agile software-like development approach, since it enables a hardware developer to patch bugs that are detected during on-chip testing or in production. Unfortunately, FPGA programmers lack bug localization tools amenable to this rapid development cycle, since past tools mainly find bugs via simulation and verification. To develop hardware bug localization tools for a rapid development cycle, a thorough understanding of the symptoms, root causes, and fixes of hardware bugs is needed. In this paper, we first study bugs in existing FPGA designs and produce a testbed of reliably-reproducible bugs. We classify the bugs according to their intrinsic properties, symptoms, and root causes. We demonstrate that many hardware bugs are comparable to software bug counterparts, and would benefit from similar techniques for bug diagnosis and repair. Based upon our findings, we build a novel collection of hybrid static/dynamic program analysis and monitoring tools for debugging FPGA designs, showing that our tools enable a software-like development cycle by effectively reducing developers' manual efforts for bug localization. @InProceedings{ASPLOS22p946, author = {Jiacheng Ma and Gefei Zuo and Kevin Loughlin and Haoyang Zhang and Andrew Quinn and Baris Kasikci}, title = {Debugging in the Brave New World of Reconfigurable Hardware}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {946--962}, doi = {10.1145/3503222.3507701}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Qureshi, Moinuddin	ASPLOS '22: "Randomized Row-Swap: Mitigating ..." Randomized Row-Swap: Mitigating Row Hammer by Breaking Spatial Correlation between Aggressor and Victim Rows Gururaj Saileshwar, Bolin Wang, Moinuddin Qureshi, and Prashant J. Nair (Georgia Institute of Technology, USA; University of British Columbia, Canada) Row Hammer is a fault-injection attack in which rapid activations to a single DRAM row causes bit-flips in nearby rows. Several recent defenses propose tracking aggressor-rows and applying mitigating action on neighboring victim rows by refreshing them. However, all such proposals using victim-focused mitigation preserve the spatial connection between victim and aggressor rows. Therefore, these proposals are susceptible to access patterns causing bit-flips in rows beyond the immediate neighbor. For example, the Half-Double attack causes bit-flips in the presence of victim-focused mitigation. We propose Randomized Row-Swap (RRS), a novel mitigation action that breaks the spatial connection between the aggressor and victim DRAM rows. This enables RRS to provide robust defense against even complex Row Hammer access patterns. RRS is an aggressor-focused mitigation that periodically swaps aggressor-rows with other randomly selected rows in memory. This limits the possible damage in any one locality within the DRAM memory. While RRS can be used with any tracking mechanism, we implement it with a Misra-Gries tracker and target a Row Hammer Threshold of 4.8K activations (similar to the state-of-the-art attacks). Our evaluations show that RRS has negligible slowdown (0.4% on average) and provides strong security guarantees for avoiding Row Hammer bit flips even under several years of continuous attack. @InProceedings{ASPLOS22p1056, author = {Gururaj Saileshwar and Bolin Wang and Moinuddin Qureshi and Prashant J. Nair}, title = {Randomized Row-Swap: Mitigating Row Hammer by Breaking Spatial Correlation between Aggressor and Victim Rows}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1056--1069}, doi = {10.1145/3503222.3507716}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced ASPLOS '22: "HAMMER: Boosting Fidelity ..." HAMMER: Boosting Fidelity of Noisy Quantum Circuits by Exploiting Hamming Behavior of Erroneous Outcomes Swamit Tannu, Poulami Das, Ramin Ayanzadeh, and Moinuddin Qureshi (University of Wisconsin-Madison, USA; Georgia Institute of Technology, USA) Quantum computers with hundreds of qubits will be available soon. Unfortunately, high device error-rates pose a significant challenge in using these near-term quantum systems to power real-world applications. Executing a program on existing quantum systems generates both correct and incorrect outcomes, but often, the output distribution is too noisy to distinguish between them. In this paper, we show that erroneous outcomes are not arbitrary but exhibit a well-defined structure when represented in the Hamming space. Our experiments on IBM and Google quantum computers show that the most frequent erroneous outcomes are more likely to be close in the Hamming space to the correct outcome. We exploit this behavior to improve the ability to infer the correct outcome. We propose Hamming Reconstruction (HAMMER), a post-processing technique that leverages the observation of Hamming behavior to reconstruct the noisy output distribution, such that the resulting distribution has higher fidelity. We evaluate HAMMER using experimental data from Google and IBM quantum computers with more than 500 unique quantum circuits and obtain an average improvement of 1.37x in the quality of solution. On Google’s publicly available QAOA datasets, we show that HAMMER sharpens the gradients on the cost function landscape. @InProceedings{ASPLOS22p529, author = {Swamit Tannu and Poulami Das and Ramin Ayanzadeh and Moinuddin Qureshi}, title = {HAMMER: Boosting Fidelity of Noisy Quantum Circuits by Exploiting Hamming Behavior of Erroneous Outcomes}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {529--540}, doi = {10.1145/3503222.3507703}, year = {2022}, } Publisher's Version
	Raiciu, Costin	ASPLOS '22: "FlexOS: Towards Flexible OS ..." FlexOS: Towards Flexible OS Isolation Hugo Lefeuvre, Vlad-Andrei Bădoiu, Alexander Jung, Stefan Lucian Teodorescu, Sebastian Rauch, Felipe Huici, Costin Raiciu, and Pierre Olivier (University of Manchester, UK; University Politehnica of Bucharest, Romania; Lancaster University, UK; unikraft.io, Germany; KIT, Germany; NEC Laboratories, Germany; Correct Networks, Romania) At design time, modern operating systems are locked in a specific safety and isolation strategy that mixes one or more hardware/software protection mechanisms (e.g. user/kernel separation); revisiting these choices after deployment requires a major refactoring effort. This rigid approach shows its limits given the wide variety of modern applications' safety/performance requirements, when new hardware isolation mechanisms are rolled out, or when existing ones break. We present FlexOS, a novel OS allowing users to easily specialize the safety and isolation strategy of an OS at compilation/deployment time instead of design time. This modular LibOS is composed of fine-grained components that can be isolated via a range of hardware protection mechanisms with various data sharing strategies and additional software hardening. The OS ships with an exploration technique helping the user navigate the vast safety/performance design space it unlocks. We implement a prototype of the system and demonstrate, for several applications (Redis/Nginx/SQLite), FlexOS' vast configuration space as well as the efficiency of the exploration technique: we evaluate 80 FlexOS configurations for Redis and show how that space can be probabilistically subset to the 5 safest ones under a given performance budget. We also show that, under equivalent configurations, FlexOS performs similarly or better than existing solutions which use fixed safety configurations. @InProceedings{ASPLOS22p467, author = {Hugo Lefeuvre and Vlad-Andrei Bădoiu and Alexander Jung and Stefan Lucian Teodorescu and Sebastian Rauch and Felipe Huici and Costin Raiciu and Pierre Olivier}, title = {FlexOS: Towards Flexible OS Isolation}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {467--482}, doi = {10.1145/3503222.3507759}, year = {2022}, } Publisher's Version Published Artifact Info Artifacts Available Artifacts Functional Results Reproduced
	Ramdas, Abishek	ASPLOS '22: "Enzian: An Open, General, ..." Enzian: An Open, General, CPU/FPGA Platform for Systems Software Research David Cock, Abishek Ramdas, Daniel Schwyn, Michael Giardino, Adam Turowski, Zhenhao He, Nora Hossle, Dario Korolija, Melissa Licciardello, Kristina Martsenko, Reto Achermann, Gustavo Alonso, and Timothy Roscoe (ETH Zurich, Switzerland; University of British Columbia, Canada) Hybrid computing platforms, comprising CPU cores and FPGA logic, are increasingly used for accelerating data-intensive workloads in cloud deployments, and are a growing topic of interest in systems research. However, from a research perspective, existing hardware platforms are limited: they are often optimized for concrete, narrow use-cases and, therefore lack the flexibility needed to explore other applications and configurations. We show that a research group can design and build a more general, open, and affordable hardware platform for hybrid systems research. The platform, Enzian, is capable of duplicating the functionality of existing CPU/FPGA systems with comparable performance but in an open, flexible system. It couples a large FPGA with a server-class CPU in an asymmetric cache-coherent NUMA system. Enzian also enables research not possible with existing hybrid platforms, through explicit access to coherence messages, extensive thermal and power instrumentation, and an open, programmable baseboard management processor. Enzian is already being used in multiple projects, is open source (both hardware and software), and available for remote use. We present the design principles of Enzian, the challenges in building it, and evaluate it with a range of existing research use-cases alongside other, more specialized platforms, as well as demonstrating research not possible on existing platforms. @InProceedings{ASPLOS22p434, author = {David Cock and Abishek Ramdas and Daniel Schwyn and Michael Giardino and Adam Turowski and Zhenhao He and Nora Hossle and Dario Korolija and Melissa Licciardello and Kristina Martsenko and Reto Achermann and Gustavo Alonso and Timothy Roscoe}, title = {Enzian: An Open, General, CPU/FPGA Platform for Systems Software Research}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {434--451}, doi = {10.1145/3503222.3507742}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Ranganathan, Parthasarathy	ASPLOS '22: "CRISP: Critical Slice Prefetching ..." CRISP: Critical Slice Prefetching Heiner Litz, Grant Ayers, and Parthasarathy Ranganathan (University of California at Santa Cruz, USA; Google, USA) The high access latency of DRAM continues to be a performance challenge for contemporary microprocessor systems. Prefetching is a well-established technique to address this problem, however, existing implemented designs fail to provide any performance benefits in the presence of irregular memory access patterns. The hardware complexity of prior techniques that can predict irregular memory accesses such as runahead execution has proven untenable for implementation in real hardware. We propose a lightweight mechanism to hide the high latency of irregular memory access patterns by leveraging criticality-based scheduling. In particular, our technique executes delinquent loads and their load slices as early as possible, hiding a significant fraction of their latency. Furthermore, we observe that the latency induced by branch mispredictions and other high latency instructions can be hidden with a similar approach. Our proposal only requires minimal hardware modifications by performing memory access classification, load and branch slice extraction, as well as priority analysis exclusively in software. As a result, our technique is feasible to implement, introducing only a simple new instruction prefix while requiring minimal modifications of the instruction scheduler. Our technique increases the IPC of memory-latency-bound applications by up to 38% and by 8.4% on average. @InProceedings{ASPLOS22p300, author = {Heiner Litz and Grant Ayers and Parthasarathy Ranganathan}, title = {CRISP: Critical Slice Prefetching}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {300--313}, doi = {10.1145/3503222.3507745}, year = {2022}, } Publisher's Version
	Rao, Gengyu	ASPLOS '22: "SparseCore: Stream ISA and ..." SparseCore: Stream ISA and Processor Specialization for Sparse Computation Gengyu Rao, Jingji Chen, Jason Yik, and Xuehai Qian (University of Southern California, USA) Computation on sparse data is becoming increasingly important for many applications. Recent sparse computation accelerators are designed for specific algorithm/application, making them inflexible with software optimizations. This paper proposes SparseCore, the first general-purpose processor extension for sparse computation that can flexibly accelerate complex code patterns and fast-evolving algorithms. We extend the instruction set architecture (ISA) to make stream or sparse vector first-class citizens, and develop efficient architectural components to support the stream ISA. The novel ISA extension intrinsically operates on streams, realizing both efficient data movement and computation. The simulation results show that SparseCore achieves significant speedups for sparse tensor computation and graph pattern computation. @InProceedings{ASPLOS22p186, author = {Gengyu Rao and Jingji Chen and Jason Yik and Xuehai Qian}, title = {SparseCore: Stream ISA and Processor Specialization for Sparse Computation}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {186--199}, doi = {10.1145/3503222.3507705}, year = {2022}, } Publisher's Version
	Rauch, Sebastian	ASPLOS '22: "FlexOS: Towards Flexible OS ..." FlexOS: Towards Flexible OS Isolation Hugo Lefeuvre, Vlad-Andrei Bădoiu, Alexander Jung, Stefan Lucian Teodorescu, Sebastian Rauch, Felipe Huici, Costin Raiciu, and Pierre Olivier (University of Manchester, UK; University Politehnica of Bucharest, Romania; Lancaster University, UK; unikraft.io, Germany; KIT, Germany; NEC Laboratories, Germany; Correct Networks, Romania) At design time, modern operating systems are locked in a specific safety and isolation strategy that mixes one or more hardware/software protection mechanisms (e.g. user/kernel separation); revisiting these choices after deployment requires a major refactoring effort. This rigid approach shows its limits given the wide variety of modern applications' safety/performance requirements, when new hardware isolation mechanisms are rolled out, or when existing ones break. We present FlexOS, a novel OS allowing users to easily specialize the safety and isolation strategy of an OS at compilation/deployment time instead of design time. This modular LibOS is composed of fine-grained components that can be isolated via a range of hardware protection mechanisms with various data sharing strategies and additional software hardening. The OS ships with an exploration technique helping the user navigate the vast safety/performance design space it unlocks. We implement a prototype of the system and demonstrate, for several applications (Redis/Nginx/SQLite), FlexOS' vast configuration space as well as the efficiency of the exploration technique: we evaluate 80 FlexOS configurations for Redis and show how that space can be probabilistically subset to the 5 safest ones under a given performance budget. We also show that, under equivalent configurations, FlexOS performs similarly or better than existing solutions which use fixed safety configurations. @InProceedings{ASPLOS22p467, author = {Hugo Lefeuvre and Vlad-Andrei Bădoiu and Alexander Jung and Stefan Lucian Teodorescu and Sebastian Rauch and Felipe Huici and Costin Raiciu and Pierre Olivier}, title = {FlexOS: Towards Flexible OS Isolation}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {467--482}, doi = {10.1145/3503222.3507759}, year = {2022}, } Publisher's Version Published Artifact Info Artifacts Available Artifacts Functional Results Reproduced
	Ravindran, Binoy	ASPLOS '22: "Adelie: Continuous Address ..." Adelie: Continuous Address Space Layout Re-randomization for Linux Drivers Ruslan Nikolaev, Hassan Nadeem, Cathlyn Stone, and Binoy Ravindran (Pennsylvania State University, USA; Virginia Tech, USA) While address space layout randomization (ASLR) has been extensively studied for user-space programs, the corresponding OS kernel's KASLR support remains very limited, making the kernel vulnerable to just-in-time (JIT) return-oriented programming (ROP) attacks. Furthermore, commodity OSs such as Linux restrict their KASLR range to 32 bits due to architectural constraints (e.g., x86-64 only supports 32-bit immediate operands for most instructions), which makes them vulnerable to even unsophisticated brute-force ROP attacks due to low entropy. Most in-kernel pointers remain static, exacerbating the problem when pointers are leaked. Adelie, our kernel defense mechanism, overcomes KASLR limitations, increases KASLR entropy, and makes successful ROP attacks on the Linux kernel much harder to achieve. First, Adelie enables the position-independent code (PIC) model so that the kernel and its modules can be placed anywhere in the 64-bit virtual address space, at any distance apart from each other. Second, Adelie implements stack re-randomization and address encryption on modules. Finally, Adelie enables efficient continuous KASLR for modules by using the PIC model to make it (almost) impossible to inject ROP gadgets through these modules regardless of gadget's origin. Since device drivers (typically compiled as modules) are often developed by third parties and are typically less tested than core OS parts, they are also often more vulnerable. By fully re-randomizing device drivers, the last two contributions together prevent most JIT ROP attacks since vulnerable modules are very likely to be a starting point of an attack. Furthermore, some OS instances in virtualized environments are specifically designated to run device drivers, where drivers are the primary target of JIT ROP attacks. Using a GCC plugin that we developed, we automatically modify different kinds of kernel modules. Since the prior art tackles only user-space programs, we solve many challenges unique to the kernel code. Our evaluation shows high efficiency of Adelie's approach: the overhead of the PIC model is completely negligible and re-randomization cost remains reasonable for typical use cases. @InProceedings{ASPLOS22p483, author = {Ruslan Nikolaev and Hassan Nadeem and Cathlyn Stone and Binoy Ravindran}, title = {Adelie: Continuous Address Space Layout Re-randomization for Linux Drivers}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {483--498}, doi = {10.1145/3503222.3507779}, year = {2022}, } Publisher's Version Published Artifact Info Artifacts Available Artifacts Functional Results Reproduced
	Reggiani, Enrico	ASPLOS '22: "BiSon-e: A Lightweight and ..." BiSon-e: A Lightweight and High-Performance Accelerator for Narrow Integer Linear Algebra Computing on the Edge Enrico Reggiani, Cristóbal Ramírez Lazo, Roger Figueras Bagué, Adrián Cristal, Mauro Olivieri, and Osman Sabri Unsal (Polytechnic University of Catalonia, Spain; Barcelona Supercomputing Center, Spain; Sapienza University of Rome, Italy) Linear algebra computational kernels based on byte and sub-byte integer data formats are at the base of many classes of applications, ranging from Deep Learning to Pattern Matching. Porting the computation of these applications from cloud to edge and mobile devices would enable significant improvements in terms of security, safety, and energy efficiency. However, despite their low memory and energy demands, their intrinsically high computational intensity makes the execution of these workloads challenging on highly resource-constrained devices. In this paper, we present BiSon-e, a novel RISC-V based architecture that accelerates linear algebra kernels based on narrow integer computations on edge processors by performing Single Instruction Multiple Data (SIMD) operations on off-the-shelf scalar Functional Units (FUs). Our novel architecture is built upon the binary segmentation technique, which allows to significantly reduce the memory footprint and the arithmetic intensity of linear algebra kernels requiring narrow data sizes. We integrate BiSon-e into a complete System-on-Chip (SoC) based on RISC-V, synthesized and Place&Routed in 65nm and 22nm technologies, introducing a negligible 0.07% area overhead with respect to the baseline architecture. Our experimental evaluation shows that, when computing the Convolution and Fully-Connected layers of the AlexNet and VGG-16 Convolutional Neural Networks (CNNs) with 8-, 4-, and 2-bit, our solution gains up to 5.6×, 13.9× and 24× in execution time compared to the scalar implementation of a single RISC-V core, and improves the energy efficiency of string matching tasks by 5× when compared to a RISC-V-based Vector Processing Unit (VPU). @InProceedings{ASPLOS22p56, author = {Enrico Reggiani and Cristóbal Ramírez Lazo and Roger Figueras Bagué and Adrián Cristal and Mauro Olivieri and Osman Sabri Unsal}, title = {BiSon-e: A Lightweight and High-Performance Accelerator for Narrow Integer Linear Algebra Computing on the Edge}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {56--69}, doi = {10.1145/3503222.3507746}, year = {2022}, } Publisher's Version
	Reidys, Benjamin	ASPLOS '22: "RSSD: Defend against Ransomware ..." RSSD: Defend against Ransomware with Hardware-Isolated Network-Storage Codesign and Post-Attack Analysis Benjamin Reidys, Peng Liu, and Jian Huang (University of Illinois at Urbana-Champaign, USA; Pennsylvania State University, USA) Encryption ransomware has become a notorious malware. It encrypts user data on storage devices like solid-state drives (SSDs) and demands a ransom to restore data for users. To bypass existing defenses, ransomware would keep evolving and performing new attack models. For instance, we identify and validate three new attacks, including (1) garbage-collection (GC) attack that exploits storage capacity and keeps writing data to trigger GC and force SSDs to release the retained data; (2) timing attack that intentionally slows down the pace of encrypting data and hides its I/O patterns to escape existing defense; (3) trimming attack that utilizes the trim command available in SSDs to physically erase data. To enhance the robustness of SSDs against these attacks, we propose RSSD, a ransomware-aware SSD. It redesigns the flash management of SSDs for enabling the hardware-assisted logging, which can conservatively retain older versions of user data and received storage operations in time order with low overhead. It also employs hardware-isolated NVMe over Ethernet to expand local storage capacity by transparently offloading the logs to remote cloud/servers in a secure manner. RSSD enables post-attack analysis by building a trusted evidence chain of storage operations to assist the investigation of ransomware attacks. We develop RSSD with a real-world SSD FPGA board. Our evaluation shows that RSSD can defend against new and future ransomware attacks, while introducing negligible performance overhead. @InProceedings{ASPLOS22p726, author = {Benjamin Reidys and Peng Liu and Jian Huang}, title = {RSSD: Defend against Ransomware with Hardware-Isolated Network-Storage Codesign and Post-Attack Analysis}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {726--739}, doi = {10.1145/3503222.3507773}, year = {2022}, } Publisher's Version
	Ren, Kui	ASPLOS '22: "EXAMINER: Automatically Locating ..." EXAMINER: Automatically Locating Inconsistent Instructions between Real Devices and CPU Emulators for ARM Muhui Jiang, Tianyi Xu, Yajin Zhou, Yufeng Hu, Ming Zhong, Lei Wu, Xiapu Luo, and Kui Ren (Hong Kong Polytechnic University, China; Zhejiang University, China) Emulators are widely used to build dynamic analysis frameworks due to its fine-grained tracing capability, full system monitoring functionality, and scalability of running on different operating systems and architectures. However, whether emulators are consistent with real devices is unknown. To understand this problem, we aim to automatically locate inconsistent instructions, which behave differently between emulators and real devices. We target the ARM architecture, which provides machine-readable specifications. Based on the specification, we propose a sufficient test case generator by designing and implementing the first symbolic execution engine for the ARM architecture specification language (ASL). We generate 2,774,649 representative instruction streams and conduct differential testing between four ARM real devices in different architecture versions (i.e., ARMv5, ARMv6, ARMv7, and ARMv8) and three state-of-the-art emulators (i.e., QEMU, Unicorn, and Angr). We locate a huge number of inconsistent instruction streams (171,858 for QEMU, 223,264 for unicorn, and 120,169 for Angr). We find that undefined implementation in ARM manual and bugs of emulators are the major causes of inconsistencies. Furthermore, we discover 12 bugs, which influence commonly used instructions (e.g., BLX). With the inconsistent instructions, we build three security applications and demonstrate the capability of these instructions on detecting emulators, anti-emulation, and anti-fuzzing. @InProceedings{ASPLOS22p846, author = {Muhui Jiang and Tianyi Xu and Yajin Zhou and Yufeng Hu and Ming Zhong and Lei Wu and Xiapu Luo and Kui Ren}, title = {EXAMINER: Automatically Locating Inconsistent Instructions between Real Devices and CPU Emulators for ARM}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {846--858}, doi = {10.1145/3503222.3507736}, year = {2022}, } Publisher's Version
	Ren, Mengfei	ASPLOS '22: "One Size Does Not Fit All: ..." One Size Does Not Fit All: Security Hardening of MIPS Embedded Systems via Static Binary Debloating for Shared Libraries Haotian Zhang, Mengfei Ren, Yu Lei, and Jiang Ming (University of Texas at Arlington, USA) Embedded systems have become prominent targets for cyberattacks. To exploit firmware’s memory corruption vulnerabilities, cybercriminals harvest reusable code gadgets from the large shared library codebase (e.g., uClibc). Unfortunately, unlike their desktop counterparts, embedded systems lack essential computing resources to enforce security hardening techniques. Recently, we have witnessed a surge of software debloating as a new defense mechanism against code-reuse attacks; it erases unused code to significantly diminish the possibilities of constructing reusable gadgets. Because of the single firmware image update style, static library debloating shows promise to fortify embedded systems without compromising performance and forward compatibility. However, static library debloating on stripped binaries (e.g., firmware’s shared libraries) is still an enormous challenge. In this paper, we show that this challenge is not insurmountable for MIPS firmware. We develop a novel system, named uTrimmer, to identify and wipe out unused basic blocks from shared libraries’ binary code, without causing additional runtime overhead or memory consumption. We propose a new method to identify address-taken blocks/functions, which further help us maintain an inter-procedural control flow graph to conservatively include library code that could be potentially used by firmware. By capturing address access patterns for position-independent code, we circumvent the challenge of determining code-pointer targets and safely eliminate unused code. We run uTrimmer to debloat shared libraries for SPEC CPU2017 benchmarks, popular firmware applications (e.g., Apache, BusyBox, and OpenSSL), and a real-world wireless router firmware image. Our experiments show that not only does uTrimmer deliver functional programs, but also it can cut the exposed code surface and eliminate various reusable code gadgets remarkably. uTrimmer’s debloating capability can compete with the static linking results. @InProceedings{ASPLOS22p255, author = {Haotian Zhang and Mengfei Ren and Yu Lei and Jiang Ming}, title = {One Size Does Not Fit All: Security Hardening of MIPS Embedded Systems via Static Binary Debloating for Shared Libraries}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {255--270}, doi = {10.1145/3503222.3507768}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Rétvári, Gábor	ASPLOS '22: "Domain Specific Run Time Optimization ..." Domain Specific Run Time Optimization for Software Data Planes Sebastiano Miano, Alireza Sanaee, Fulvio Risso, Gábor Rétvári, and Gianni Antichi (Queen Mary University of London, UK; Politecnico di Torino, Italy; Budapest University of Technology and Economics, Hungary) State-of-the-art approaches to design, develop and optimize software packet-processing programs are based on static compilation: the compiler's input is a description of the forwarding plane semantics and the output is a binary that can accommodate any control plane configuration or input traffic. In this paper, we demonstrate that tracking control plane actions and packet-level traffic dynamics at run time opens up new opportunities for code specialization. We present Morpheus, a system working alongside static compilers that continuously optimizes the targeted networking code. We introduce a number of new techniques, from static code analysis to adaptive code instrumentation, and we implement a toolbox of domain specific optimizations that are not restricted to a specific data plane framework or programming language. We apply Morpheus to several eBPF and DPDK programs including Katran, Facebook's production-grade load balancer. We compare Morpheus against state-of-the-art optimization frameworks and show that it can bring up to 2x throughput improvement, while halving the 99th percentile latency. @InProceedings{ASPLOS22p1148, author = {Sebastiano Miano and Alireza Sanaee and Fulvio Risso and Gábor Rétvári and Gianni Antichi}, title = {Domain Specific Run Time Optimization for Software Data Planes}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1148--1164}, doi = {10.1145/3503222.3507769}, year = {2022}, } Publisher's Version Published Artifact Info Artifacts Available Artifacts Functional Results Reproduced
	Rigger, Manuel	ASPLOS '22: "Finding Missed Optimizations ..." Finding Missed Optimizations through the Lens of Dead Code Elimination Theodoros Theodoridis, Manuel Rigger, and Zhendong Su (ETH Zurich, Switzerland) Compilers are foundational software development tools and incorporate increasingly sophisticated optimizations. Due to their complexity, it is difficult to systematically identify opportunities for improving them. Indeed, the automatic discovery of missed optimizations has been an important and significant challenge. The few existing approaches either cannot accurately pinpoint missed optimizations or target only specific analyses. This paper tackles this challenge by introducing a novel, effective approach that --- in a simple and general manner --- automatically identifies a wide range of missed optimizations. Our core insight is to leverage dead code elimination (DCE) to both analyze how well compilers optimize code and identify missed optimizations: (1) insert "optimization markers" in the basic blocks of a given program, (2) compute the program's live/dead basic blocks using the "optimization markers", and (3) identify missed optimizations from how well compilers eliminate dead blocks. We essentially exploit that, since DCE heavily depends on the rest of the optimization pipeline, through the lens of DCE, one can systematically quantify how well compilers optimize code. We conduct an extensive analysis of GCC and LLVM using our approach, which (1) provides quantitative and qualitative insights regarding their optimization capabilities, and (2) uncovers a diverse set of missed optimizations. Our results also lead to 84 bug reports for GCC and LLVM, of which 62 have already been confirmed or fixed, demonstrating our work's strong practical utility. We expect that the simplicity and generality of our approach will make it widely applicable for understanding compiler performance and finding missed optimizations. This work opens and initiates this promising direction. @InProceedings{ASPLOS22p697, author = {Theodoros Theodoridis and Manuel Rigger and Zhendong Su}, title = {Finding Missed Optimizations through the Lens of Dead Code Elimination}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {697--709}, doi = {10.1145/3503222.3507764}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Risso, Fulvio	ASPLOS '22: "Domain Specific Run Time Optimization ..." Domain Specific Run Time Optimization for Software Data Planes Sebastiano Miano, Alireza Sanaee, Fulvio Risso, Gábor Rétvári, and Gianni Antichi (Queen Mary University of London, UK; Politecnico di Torino, Italy; Budapest University of Technology and Economics, Hungary) State-of-the-art approaches to design, develop and optimize software packet-processing programs are based on static compilation: the compiler's input is a description of the forwarding plane semantics and the output is a binary that can accommodate any control plane configuration or input traffic. In this paper, we demonstrate that tracking control plane actions and packet-level traffic dynamics at run time opens up new opportunities for code specialization. We present Morpheus, a system working alongside static compilers that continuously optimizes the targeted networking code. We introduce a number of new techniques, from static code analysis to adaptive code instrumentation, and we implement a toolbox of domain specific optimizations that are not restricted to a specific data plane framework or programming language. We apply Morpheus to several eBPF and DPDK programs including Katran, Facebook's production-grade load balancer. We compare Morpheus against state-of-the-art optimization frameworks and show that it can bring up to 2x throughput improvement, while halving the 99th percentile latency. @InProceedings{ASPLOS22p1148, author = {Sebastiano Miano and Alireza Sanaee and Fulvio Risso and Gábor Rétvári and Gianni Antichi}, title = {Domain Specific Run Time Optimization for Software Data Planes}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1148--1164}, doi = {10.1145/3503222.3507769}, year = {2022}, } Publisher's Version Published Artifact Info Artifacts Available Artifacts Functional Results Reproduced
	Robson, Eyes	ASPLOS '22: "ProSE: The Architecture and ..." ProSE: The Architecture and Design of a Protein Discovery Engine Eyes Robson, Ceyu Xu, and Lisa Wu Wills (University of California at Berkeley, USA; Duke University, USA) Protein language models have enabled breakthrough approaches to protein structure prediction, function annotation, and drug discovery. A primary limitation to the widespread adoption of these powerful models is the high computational cost associated with the training and inference of these models, especially at longer sequence lengths. We present the architecture, microarchitecture, and hardware implementation of a protein design and discovery accelerator, ProSE (Protein Systolic Engine). ProSE has a collection of custom heterogeneous systolic arrays and special functions that process transfer learning model inferences efficiently. The architecture marries SIMD-style computations with systolic array architectures, optimizing coarse-grained operation sequences across model layers to achieve efficiency without sacrificing generality. ProSE performs Protein BERT inference at up to 6.9× speedup and 48× power efficiency (performance/Watt) compared to one NVIDIA A100 GPU. ProSE achieves up to 5.5 × (12.7×) speedup and 173× (249×) power efficiency compared to TPUv3 (TPUv2). @InProceedings{ASPLOS22p655, author = {Eyes Robson and Ceyu Xu and Lisa Wu Wills}, title = {ProSE: The Architecture and Design of a Protein Discovery Engine}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {655--668}, doi = {10.1145/3503222.3507722}, year = {2022}, } Publisher's Version
	Roitzsch, Michael	ASPLOS '22: "Efficient and Scalable Core ..." Efficient and Scalable Core Multiplexing with M³v Nils Asmussen, Sebastian Haas, Carsten Weinhold, Till Miemietz, and Michael Roitzsch (Barkhausen Institut, Germany) The M³ system (ASPLOS ’16) proposed a hardware/software co-design that simplifies integration between general-purpose cores and special-purpose accelerators, allowing users to easily utilize them in a unified manner. M³ is a tiled architecture, whose tiles (cores and accelerators) are partitioned between applications, such that each tile is dedicated to its own application. The M³x system (ATC ’19) extended M³ by trading off some isolation to enable coarse-grained multiplexing of tiles among multiple applications. With M³x, if source tile t₁ runs code of application p and sends a message m to destination tile t₂ while t₂ is currently not associated with p, then m is forwarded to the right place through a “slow path”, via some special OS tile. In this paper, we present M³v, which extends M³x by further trading off some isolation between applications to support “fast path” communication that does not require the said OS tile’s involvement. Thus, with M³v, a tile can be efficiently multiplexed between applications provided it is a general-purpose core. M³v achieves this goal by 1) adding a local multiplexer to each such core, and by 2) virtualizing the core’s hardware component responsible for cross-tile communications. We prototype M³v using RISC-V cores on an FPGA platform and show that it significantly outperforms M³x and may achieve competitive performance to Linux. @InProceedings{ASPLOS22p452, author = {Nils Asmussen and Sebastian Haas and Carsten Weinhold and Till Miemietz and Michael Roitzsch}, title = {Efficient and Scalable Core Multiplexing with M³v}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {452--466}, doi = {10.1145/3503222.3507741}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Root, Alexander J.	ASPLOS '22: "Vector Instruction Selection ..." Vector Instruction Selection for Digital Signal Processors using Program Synthesis Maaz Bin Safeer Ahmad, Alexander J. Root, Andrew Adams, Shoaib Kamil, and Alvin Cheung (Adobe, USA; Massachusetts Institute of Technology, USA; University of California at Berkeley, USA) Instruction selection, whereby input code represented in an intermediate representation is translated into executable instructions from the target platform, is often the most target-dependent component in optimizing compilers. Current approaches include pattern matching, which is brittle and tedious to design, or search-based methods, which are limited by scalability of the search algorithm. In this paper, we propose a new algorithm that first abstracts the target platform instructions into high-level uber-instructions, with each uber-instruction unifying multiple concrete instructions from the target platform. Program synthesis is used to lift input code sequences into semantically equivalent sequences of uber-instructions and then to lower from uber-instructions to machine code. Using 21 real-world benchmarks, we show that our synthesis-based instruction selection algorithm can generate instruction sequences for a hardware target, with the synthesized code performing up to 2.1x faster as compared to code generated by a professionally-developed optimizing compiler for the same platform. @InProceedings{ASPLOS22p1004, author = {Maaz Bin Safeer Ahmad and Alexander J. Root and Andrew Adams and Shoaib Kamil and Alvin Cheung}, title = {Vector Instruction Selection for Digital Signal Processors using Program Synthesis}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1004--1016}, doi = {10.1145/3503222.3507714}, year = {2022}, } Publisher's Version
	Roscoe, Timothy	ASPLOS '22: "Enzian: An Open, General, ..." Enzian: An Open, General, CPU/FPGA Platform for Systems Software Research David Cock, Abishek Ramdas, Daniel Schwyn, Michael Giardino, Adam Turowski, Zhenhao He, Nora Hossle, Dario Korolija, Melissa Licciardello, Kristina Martsenko, Reto Achermann, Gustavo Alonso, and Timothy Roscoe (ETH Zurich, Switzerland; University of British Columbia, Canada) Hybrid computing platforms, comprising CPU cores and FPGA logic, are increasingly used for accelerating data-intensive workloads in cloud deployments, and are a growing topic of interest in systems research. However, from a research perspective, existing hardware platforms are limited: they are often optimized for concrete, narrow use-cases and, therefore lack the flexibility needed to explore other applications and configurations. We show that a research group can design and build a more general, open, and affordable hardware platform for hybrid systems research. The platform, Enzian, is capable of duplicating the functionality of existing CPU/FPGA systems with comparable performance but in an open, flexible system. It couples a large FPGA with a server-class CPU in an asymmetric cache-coherent NUMA system. Enzian also enables research not possible with existing hybrid platforms, through explicit access to coherence messages, extensive thermal and power instrumentation, and an open, programmable baseboard management processor. Enzian is already being used in multiple projects, is open source (both hardware and software), and available for remote use. We present the design principles of Enzian, the challenges in building it, and evaluate it with a range of existing research use-cases alongside other, more specialized platforms, as well as demonstrating research not possible on existing platforms. @InProceedings{ASPLOS22p434, author = {David Cock and Abishek Ramdas and Daniel Schwyn and Michael Giardino and Adam Turowski and Zhenhao He and Nora Hossle and Dario Korolija and Melissa Licciardello and Kristina Martsenko and Reto Achermann and Gustavo Alonso and Timothy Roscoe}, title = {Enzian: An Open, General, CPU/FPGA Platform for Systems Software Research}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {434--451}, doi = {10.1145/3503222.3507742}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Roy, Rohan Basu	ASPLOS '22: "IceBreaker: Warming Serverless ..." IceBreaker: Warming Serverless Functions Better with Heterogeneity Rohan Basu Roy, Tirthak Patel, and Devesh Tiwari (Northeastern University, USA) Serverless computing, an emerging computing model, relies on "warming up" functions prior to its anticipated execution for faster and cost-effective service to users. Unfortunately, warming up functions can be inaccurate and incur prohibitively expensive cost during the warmup period (i.e., keep-alive cost). In this paper, we introduce IceBreaker, a novel technique that reduces the service time and the "keep-alive" cost by composing a system with heterogeneous nodes (costly and cheaper). IceBreaker does so by dynamically determining the cost-effective node type to warm up a function based on the function's time-varying probability of the next invocation. By employing heterogeneity, IceBreaker allows for more number of nodes under the same cost budget and hence, keeps more number of functions warm and reduces the wait time during high load. Our real-system evaluation confirms that IceBreaker reduces the overall keep-alive cost by 45% and execution time by 27% using representative serverless applications and industry-grade workload trace. IceBreaker is the first technique to employ and leverage the idea of mixing expensive and cheaper nodes to improve both service time and keep-alive cost for serverless functions -- opening up a new research avenue of serverless computing on heterogeneous servers for researchers and practitioners. @InProceedings{ASPLOS22p753, author = {Rohan Basu Roy and Tirthak Patel and Devesh Tiwari}, title = {IceBreaker: Warming Serverless Functions Better with Heterogeneity}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {753--767}, doi = {10.1145/3503222.3507750}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Rucker, Alexander	ASPLOS '22: "Taurus: A Data Plane Architecture ..." Taurus: A Data Plane Architecture for Per-Packet ML Tushar Swamy, Alexander Rucker, Muhammad Shahbaz, Ishan Gaur, and Kunle Olukotun (Stanford University, USA; Purdue University, USA) Emerging applications---cloud computing, the internet of things, and augmented/virtual reality---demand responsive, secure, and scalable datacenter networks. These networks currently implement simple, per-packet, data-plane heuristics (e.g., ECMP and sketches) under a slow, millisecond-latency control plane that runs data-driven performance and security policies. However, to meet applications' service-level objectives (SLOs) in a modern data center, networks must bridge the gap between line-rate, per-packet execution and complex decision making. In this work, we present the design and implementation of Taurus, a data plane for line-rate inference. Taurus adds custom hardware based on a flexible, parallel-patterns (MapReduce) abstraction to programmable network devices, such as switches and NICs; this new hardware uses pipelined SIMD parallelism to enable per-packet MapReduce operations (e.g., inference). Our evaluation of a Taurus switch ASIC---supporting several real-world models---shows that Taurus operates orders of magnitude faster than a server-based control plane while increasing area by 3.8% and latency for line-rate ML models by up to 221 ns. Furthermore, our Taurus FPGA prototype achieves full model accuracy and detects two orders of magnitude more events than a state-of-the-art control-plane anomaly-detection system. @InProceedings{ASPLOS22p1099, author = {Tushar Swamy and Alexander Rucker and Muhammad Shahbaz and Ishan Gaur and Kunle Olukotun}, title = {Taurus: A Data Plane Architecture for Per-Packet ML}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1099--1114}, doi = {10.1145/3503222.3507726}, year = {2022}, } Publisher's Version Published Artifact Info Artifacts Available
	Saarikivi, Olli	ASPLOS '22: "Breaking the Computation and ..." Breaking the Computation and Communication Abstraction Barrier in Distributed Machine Learning Workloads Abhinav Jangda, Jun Huang, Guodong Liu, Amir Hossein Nodehi Sabet, Saeed Maleki, Youshan Miao, Madanlal Musuvathi, Todd Mytkowicz, and Olli Saarikivi (University of Massachusetts at Amherst, USA; Ohio State University, USA; Chinese Academy of Sciences, China; University of California at Riverside, USA; Microsoft Research, USA; Microsoft Research, China) Recent trends towards large machine learning models require both training and inference tasks to be distributed. Considering the huge cost of training these models, it is imperative to unlock optimizations in computation and communication to obtain best performance. However, the current logical separation between computation and communication kernels in machine learning frameworks misses optimization opportunities across this barrier. Breaking this abstraction can provide many optimizations to improve the performance of distributed workloads. However, manually applying these optimizations requires modifying the underlying computation and communication libraries for each scenario, which is both time consuming and error-prone. Therefore, we present CoCoNet, which contains (i) a domain specific language to express a distributed machine learning program in the form of computation and communication operations, (ii) a set of semantics preserving transformations to optimize the program, and (iii) a compiler to generate jointly optimized communication and computation GPU kernels. Providing both computation and communication as first class constructs allows users to work on a high-level abstraction and apply powerful optimizations, such as fusion or overlapping of communication and computation. CoCoNet enabled us to optimize data-, model- and pipeline-parallel workloads in large language models with only a few lines of code. Our experiments show that CoCoNet significantly outperforms state-of-the-art distributed machine learning implementations. @InProceedings{ASPLOS22p402, author = {Abhinav Jangda and Jun Huang and Guodong Liu and Amir Hossein Nodehi Sabet and Saeed Maleki and Youshan Miao and Madanlal Musuvathi and Todd Mytkowicz and Olli Saarikivi}, title = {Breaking the Computation and Communication Abstraction Barrier in Distributed Machine Learning Workloads}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {402--416}, doi = {10.1145/3503222.3507778}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional
	Sabet, Amir Hossein Nodehi	ASPLOS '22: "Breaking the Computation and ..." Breaking the Computation and Communication Abstraction Barrier in Distributed Machine Learning Workloads Abhinav Jangda, Jun Huang, Guodong Liu, Amir Hossein Nodehi Sabet, Saeed Maleki, Youshan Miao, Madanlal Musuvathi, Todd Mytkowicz, and Olli Saarikivi (University of Massachusetts at Amherst, USA; Ohio State University, USA; Chinese Academy of Sciences, China; University of California at Riverside, USA; Microsoft Research, USA; Microsoft Research, China) Recent trends towards large machine learning models require both training and inference tasks to be distributed. Considering the huge cost of training these models, it is imperative to unlock optimizations in computation and communication to obtain best performance. However, the current logical separation between computation and communication kernels in machine learning frameworks misses optimization opportunities across this barrier. Breaking this abstraction can provide many optimizations to improve the performance of distributed workloads. However, manually applying these optimizations requires modifying the underlying computation and communication libraries for each scenario, which is both time consuming and error-prone. Therefore, we present CoCoNet, which contains (i) a domain specific language to express a distributed machine learning program in the form of computation and communication operations, (ii) a set of semantics preserving transformations to optimize the program, and (iii) a compiler to generate jointly optimized communication and computation GPU kernels. Providing both computation and communication as first class constructs allows users to work on a high-level abstraction and apply powerful optimizations, such as fusion or overlapping of communication and computation. CoCoNet enabled us to optimize data-, model- and pipeline-parallel workloads in large language models with only a few lines of code. Our experiments show that CoCoNet significantly outperforms state-of-the-art distributed machine learning implementations. @InProceedings{ASPLOS22p402, author = {Abhinav Jangda and Jun Huang and Guodong Liu and Amir Hossein Nodehi Sabet and Saeed Maleki and Youshan Miao and Madanlal Musuvathi and Todd Mytkowicz and Olli Saarikivi}, title = {Breaking the Computation and Communication Abstraction Barrier in Distributed Machine Learning Workloads}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {402--416}, doi = {10.1145/3503222.3507778}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional
	Saileshwar, Gururaj	ASPLOS '22: "Randomized Row-Swap: Mitigating ..." Randomized Row-Swap: Mitigating Row Hammer by Breaking Spatial Correlation between Aggressor and Victim Rows Gururaj Saileshwar, Bolin Wang, Moinuddin Qureshi, and Prashant J. Nair (Georgia Institute of Technology, USA; University of British Columbia, Canada) Row Hammer is a fault-injection attack in which rapid activations to a single DRAM row causes bit-flips in nearby rows. Several recent defenses propose tracking aggressor-rows and applying mitigating action on neighboring victim rows by refreshing them. However, all such proposals using victim-focused mitigation preserve the spatial connection between victim and aggressor rows. Therefore, these proposals are susceptible to access patterns causing bit-flips in rows beyond the immediate neighbor. For example, the Half-Double attack causes bit-flips in the presence of victim-focused mitigation. We propose Randomized Row-Swap (RRS), a novel mitigation action that breaks the spatial connection between the aggressor and victim DRAM rows. This enables RRS to provide robust defense against even complex Row Hammer access patterns. RRS is an aggressor-focused mitigation that periodically swaps aggressor-rows with other randomly selected rows in memory. This limits the possible damage in any one locality within the DRAM memory. While RRS can be used with any tracking mechanism, we implement it with a Misra-Gries tracker and target a Row Hammer Threshold of 4.8K activations (similar to the state-of-the-art attacks). Our evaluations show that RRS has negligible slowdown (0.4% on average) and provides strong security guarantees for avoiding Row Hammer bit flips even under several years of continuous attack. @InProceedings{ASPLOS22p1056, author = {Gururaj Saileshwar and Bolin Wang and Moinuddin Qureshi and Prashant J. Nair}, title = {Randomized Row-Swap: Mitigating Row Hammer by Breaking Spatial Correlation between Aggressor and Victim Rows}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1056--1069}, doi = {10.1145/3503222.3507716}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Sanaee, Alireza	ASPLOS '22: "Domain Specific Run Time Optimization ..." Domain Specific Run Time Optimization for Software Data Planes Sebastiano Miano, Alireza Sanaee, Fulvio Risso, Gábor Rétvári, and Gianni Antichi (Queen Mary University of London, UK; Politecnico di Torino, Italy; Budapest University of Technology and Economics, Hungary) State-of-the-art approaches to design, develop and optimize software packet-processing programs are based on static compilation: the compiler's input is a description of the forwarding plane semantics and the output is a binary that can accommodate any control plane configuration or input traffic. In this paper, we demonstrate that tracking control plane actions and packet-level traffic dynamics at run time opens up new opportunities for code specialization. We present Morpheus, a system working alongside static compilers that continuously optimizes the targeted networking code. We introduce a number of new techniques, from static code analysis to adaptive code instrumentation, and we implement a toolbox of domain specific optimizations that are not restricted to a specific data plane framework or programming language. We apply Morpheus to several eBPF and DPDK programs including Katran, Facebook's production-grade load balancer. We compare Morpheus against state-of-the-art optimization frameworks and show that it can bring up to 2x throughput improvement, while halving the 99th percentile latency. @InProceedings{ASPLOS22p1148, author = {Sebastiano Miano and Alireza Sanaee and Fulvio Risso and Gábor Rétvári and Gianni Antichi}, title = {Domain Specific Run Time Optimization for Software Data Planes}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1148--1164}, doi = {10.1145/3503222.3507769}, year = {2022}, } Publisher's Version Published Artifact Info Artifacts Available Artifacts Functional Results Reproduced
	Sandberg, Andreas	ASPLOS '22: "Every Walk’s a Hit: Making ..." Every Walk’s a Hit: Making Page Walks Single-Access Cache Hits Chang Hyun Park, Ilias Vougioukas, Andreas Sandberg, and David Black-Schaffer (Uppsala University, Sweden; Arm Research, UK) As memory capacity has outstripped TLB coverage, large data applications suffer from frequent page table walks. We investigate two complementary techniques for addressing this cost: reducing the number of accesses required and reducing the latency of each access. The first approach is accomplished by opportunistically "flattening" the page table: merging two levels of traditional 4 KB page table nodes into a single 2 MB node, thereby reducing the table's depth and the number of indirections required to traverse it. The second is accomplished by biasing the cache replacement algorithm to keep page table entries during periods of high TLB miss rates, as these periods also see high data miss rates and are therefore more likely to benefit from having the smaller page table in the cache than to suffer from increased data cache misses. We evaluate these approaches for both native and virtualized systems and across a range of realistic memory fragmentation scenarios, describe the limited changes needed in our kernel implementation and hardware design, identify and address challenges related to self-referencing page tables and kernel memory allocation, and compare results across server and mobile systems using both academic and industrial simulators for robustness. We find that flattening does reduce the number of accesses required on a page walk (to 1.0), but its performance impact (+2.3%) is small due to Page Walker Caches (already 1.5 accesses). Prioritizing caching has a larger effect (+6.8%), and the combination improves performance by +9.2%. Flattening is more effective on virtualized systems (4.4 to 2.8 accesses, +7.1% performance), due to 2D page walks. By combining the two techniques we demonstrate a state-of-the-art +14.0% performance gain and -8.7% dynamic cache energy and -4.7% dynamic DRAM energy for virtualized execution with very simple hardware and software changes. @InProceedings{ASPLOS22p128, author = {Chang Hyun Park and Ilias Vougioukas and Andreas Sandberg and David Black-Schaffer}, title = {Every Walk’s a Hit: Making Page Walks Single-Access Cache Hits}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {128--141}, doi = {10.1145/3503222.3507718}, year = {2022}, } Publisher's Version
	Sanouillet, Blaise	ASPLOS '22: "TMO: Transparent Memory Offloading ..." TMO: Transparent Memory Offloading in Datacenters Johannes Weiner, Niket Agarwal, Dan Schatzberg, Leon Yang, Hao Wang, Blaise Sanouillet, Bikash Sharma, Tejun Heo, Mayank Jain, Chunqiang Tang, and Dimitrios Skarlatos (Meta, USA; Carnegie Mellon University, USA) The unrelenting growth of the memory needs of emerging datacenter applications, along with ever increasing cost and volatility of DRAM prices, has led to DRAM being a major infrastructure expense. Alternative technologies, such as NVMe SSDs and upcoming NVM devices, offer higher capacity than DRAM at a fraction of the cost and power. One promising approach is to transparently offload colder memory to cheaper memory technologies via kernel or hypervisor techniques. The key challenge, however, is to develop a datacenter-scale solution that is robust in dealing with diverse workloads and large performance variance of different offload devices such as compressed memory, SSD, and NVM. This paper presents TMO, Meta’s transparent memory offloading solution for heterogeneous datacenter environments. TMO introduces a new Linux kernel mechanism that directly measures in realtime the lost work due to resource shortage across CPU, memory, and I/O. Guided by this information and without any prior application knowledge, TMO automatically adjusts how much memory to offload to heterogeneous devices (e.g., compressed memory or SSD) according to the device’s performance characteristics and the application’s sensitivity to memory-access slowdown. TMO holistically identifies offloading opportunities from not only the application containers but also the sidecar containers that provide infrastructure-level functions. To maximize memory savings, TMO targets both anonymous memory and file cache, and balances the swap-in rate of anonymous memory and the reload rate of file pages that were recently evicted from the file cache. TMO has been running in production for more than a year, and has saved between 20-32% of the total memory across millions of servers in our large datacenter fleet. We have successfully upstreamed TMO into the Linux kernel. @InProceedings{ASPLOS22p609, author = {Johannes Weiner and Niket Agarwal and Dan Schatzberg and Leon Yang and Hao Wang and Blaise Sanouillet and Bikash Sharma and Tejun Heo and Mayank Jain and Chunqiang Tang and Dimitrios Skarlatos}, title = {TMO: Transparent Memory Offloading in Datacenters}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {609--621}, doi = {10.1145/3503222.3507731}, year = {2022}, } Publisher's Version
	Schatzberg, Dan	ASPLOS '22: "IOCost: Block IO Control for ..." IOCost: Block IO Control for Containers in Datacenters Tejun Heo, Dan Schatzberg, Andrew Newell, Song Liu, Saravanan Dhakshinamurthy, Iyswarya Narayanan, Josef Bacik, Chris Mason, Chunqiang Tang, and Dimitrios Skarlatos (Meta, USA; Carnegie Mellon University, USA) Resource isolation is a fundamental requirement in datacenter environments. However, our production experience in Meta’s large-scale datacenters shows that existing IO control mechanisms for block storage are inadequate in containerized environments. IO control needs to provide proportional resources to containers while taking into account the hardware heterogeneity of storage devices and the idiosyncrasies of the workloads deployed in datacenters. The speed of modern SSDs requires IO control to execute with low-overheads. Furthermore, IO control should strive for work conservation, take into account the interactions with the memory management subsystem, and avoid priority inversions that lead to isolation failures. To address these challenges, this paper presents IOCost, an IO control solution that is designed for containerized environments and provides scalable, work-conserving, and low-overhead IO control for heterogeneous storage devices and diverse workloads in datacenters. IOCost performs offline profiling to build a device model and uses it to estimate device occupancy of each IO request. To minimize runtime overhead, it separates IO control into a fast per-IO issue path and a slower periodic planning path. A novel work-conserving budget donation algorithm enables containers to dynamically share unused budget. We have deployed IOCost across the entirety of Meta’s datacenters comprised of millions of ma- chines, upstreamed IOCost to the Linux kernel, and open-sourced our device-profiling tools. IOCost has been running in production for two years, providing IO control for Meta’s fleet. We describe the design of IOCost and share our experience deploying it at scale. @InProceedings{ASPLOS22p595, author = {Tejun Heo and Dan Schatzberg and Andrew Newell and Song Liu and Saravanan Dhakshinamurthy and Iyswarya Narayanan and Josef Bacik and Chris Mason and Chunqiang Tang and Dimitrios Skarlatos}, title = {IOCost: Block IO Control for Containers in Datacenters}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {595--608}, doi = {10.1145/3503222.3507727}, year = {2022}, } Publisher's Version ASPLOS '22: "TMO: Transparent Memory Offloading ..." TMO: Transparent Memory Offloading in Datacenters Johannes Weiner, Niket Agarwal, Dan Schatzberg, Leon Yang, Hao Wang, Blaise Sanouillet, Bikash Sharma, Tejun Heo, Mayank Jain, Chunqiang Tang, and Dimitrios Skarlatos (Meta, USA; Carnegie Mellon University, USA) The unrelenting growth of the memory needs of emerging datacenter applications, along with ever increasing cost and volatility of DRAM prices, has led to DRAM being a major infrastructure expense. Alternative technologies, such as NVMe SSDs and upcoming NVM devices, offer higher capacity than DRAM at a fraction of the cost and power. One promising approach is to transparently offload colder memory to cheaper memory technologies via kernel or hypervisor techniques. The key challenge, however, is to develop a datacenter-scale solution that is robust in dealing with diverse workloads and large performance variance of different offload devices such as compressed memory, SSD, and NVM. This paper presents TMO, Meta’s transparent memory offloading solution for heterogeneous datacenter environments. TMO introduces a new Linux kernel mechanism that directly measures in realtime the lost work due to resource shortage across CPU, memory, and I/O. Guided by this information and without any prior application knowledge, TMO automatically adjusts how much memory to offload to heterogeneous devices (e.g., compressed memory or SSD) according to the device’s performance characteristics and the application’s sensitivity to memory-access slowdown. TMO holistically identifies offloading opportunities from not only the application containers but also the sidecar containers that provide infrastructure-level functions. To maximize memory savings, TMO targets both anonymous memory and file cache, and balances the swap-in rate of anonymous memory and the reload rate of file pages that were recently evicted from the file cache. TMO has been running in production for more than a year, and has saved between 20-32% of the total memory across millions of servers in our large datacenter fleet. We have successfully upstreamed TMO into the Linux kernel. @InProceedings{ASPLOS22p609, author = {Johannes Weiner and Niket Agarwal and Dan Schatzberg and Leon Yang and Hao Wang and Blaise Sanouillet and Bikash Sharma and Tejun Heo and Mayank Jain and Chunqiang Tang and Dimitrios Skarlatos}, title = {TMO: Transparent Memory Offloading in Datacenters}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {609--621}, doi = {10.1145/3503222.3507731}, year = {2022}, } Publisher's Version
	Schwyn, Daniel	ASPLOS '22: "Enzian: An Open, General, ..." Enzian: An Open, General, CPU/FPGA Platform for Systems Software Research David Cock, Abishek Ramdas, Daniel Schwyn, Michael Giardino, Adam Turowski, Zhenhao He, Nora Hossle, Dario Korolija, Melissa Licciardello, Kristina Martsenko, Reto Achermann, Gustavo Alonso, and Timothy Roscoe (ETH Zurich, Switzerland; University of British Columbia, Canada) Hybrid computing platforms, comprising CPU cores and FPGA logic, are increasingly used for accelerating data-intensive workloads in cloud deployments, and are a growing topic of interest in systems research. However, from a research perspective, existing hardware platforms are limited: they are often optimized for concrete, narrow use-cases and, therefore lack the flexibility needed to explore other applications and configurations. We show that a research group can design and build a more general, open, and affordable hardware platform for hybrid systems research. The platform, Enzian, is capable of duplicating the functionality of existing CPU/FPGA systems with comparable performance but in an open, flexible system. It couples a large FPGA with a server-class CPU in an asymmetric cache-coherent NUMA system. Enzian also enables research not possible with existing hybrid platforms, through explicit access to coherence messages, extensive thermal and power instrumentation, and an open, programmable baseboard management processor. Enzian is already being used in multiple projects, is open source (both hardware and software), and available for remote use. We present the design principles of Enzian, the challenges in building it, and evaluate it with a range of existing research use-cases alongside other, more specialized platforms, as well as demonstrating research not possible on existing platforms. @InProceedings{ASPLOS22p434, author = {David Cock and Abishek Ramdas and Daniel Schwyn and Michael Giardino and Adam Turowski and Zhenhao He and Nora Hossle and Dario Korolija and Melissa Licciardello and Kristina Martsenko and Reto Achermann and Gustavo Alonso and Timothy Roscoe}, title = {Enzian: An Open, General, CPU/FPGA Platform for Systems Software Research}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {434--451}, doi = {10.1145/3503222.3507742}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Senol Cali, Damla	ASPLOS '22: "GenStore: A High-Performance ..." GenStore: A High-Performance In-Storage Processing System for Genome Sequence Analysis Nika Mansouri Ghiasi, Jisung Park, Harun Mustafa, Jeremie Kim, Ataberk Olgun, Arvid Gollwitzer, Damla Senol Cali, Can Firtina, Haiyu Mao, Nour Almadhoun Alserr, Rachata Ausavarungnirun, Nandita Vijaykumar, Mohammed Alser, and Onur Mutlu (ETH Zurich, Switzerland; Bionano Genomics, USA; King Mongkut's University of Technology North Bangkok, Thailand; University of Toronto, Canada) Read mapping is a fundamental step in many genomics applications. It is used to identify potential matches and differences between fragments (called reads) of a sequenced genome and an already known genome (called a reference genome). Read mapping is costly because it needs to perform approximate string matching (ASM) on large amounts of data. To address the computational challenges in genome analysis, many prior works propose various approaches such as accurate filters that select the reads within a dataset of genomic reads (called a read set) that must undergo expensive computation, efficient heuristics, and hardware acceleration. While effective at reducing the amount of expensive computation, all such approaches still require the costly movement of a large amount of data from storage to the rest of the system, which can significantly lower the end-to-end performance of read mapping in conventional and emerging genomics systems. We propose GenStore, the first in-storage processing system designed for genome sequence analysis that greatly reduces both data movement and computational overheads of genome sequence analysis by exploiting low-cost and accurate in-storage filters. GenStore leverages hardware/software co-design to address the challenges of in-storage processing, supporting reads with 1) different properties such as read lengths and error rates, which highly depend on the sequencing technology, and 2) different degrees of genetic variation compared to the reference genome, which highly depends on the genomes that are being compared. Through rigorous analysis of read mapping processes of reads with different properties and degrees of genetic variation, we meticulously design low-cost hardware accelerators and data/computation flows inside a NAND flash-based solid-state drive (SSD). Our evaluation using a wide range of real genomic datasets shows that GenStore, when implemented in three modern NAND flash-based SSDs, significantly improves the read mapping performance of state-of-the-art software (hardware) baselines by 2.07-6.05× (1.52-3.32×) for read sets with high similarity to the reference genome and 1.45-33.63× (2.70-19.2×) for read sets with low similarity to the reference genome. @InProceedings{ASPLOS22p635, author = {Nika Mansouri Ghiasi and Jisung Park and Harun Mustafa and Jeremie Kim and Ataberk Olgun and Arvid Gollwitzer and Damla Senol Cali and Can Firtina and Haiyu Mao and Nour Almadhoun Alserr and Rachata Ausavarungnirun and Nandita Vijaykumar and Mohammed Alser and Onur Mutlu}, title = {GenStore: A High-Performance In-Storage Processing System for Genome Sequence Analysis}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {635--654}, doi = {10.1145/3503222.3507702}, year = {2022}, } Publisher's Version
	Sethi, Geet	ASPLOS '22: "RecShard: Statistical Feature-Based ..." RecShard: Statistical Feature-Based Memory Optimization for Industry-Scale Neural Recommendation Geet Sethi, Bilge Acun, Niket Agarwal, Christos Kozyrakis, Caroline Trippel, and Carole-Jean Wu (Stanford University, USA; Meta, USA) We propose RecShard, a fine-grained embedding table (EMB) partitioning and placement technique for deep learning recommendation models (DLRMs). RecShard is designed based on two key observations. First, not all EMBs are equal, nor all rows within an EMB are equal in terms of access patterns. EMBs exhibit distinct memory characteristics, providing performance optimization opportunities for intelligent EMB partitioning and placement across a tiered memory hierarchy. Second, in modern DLRMs, EMBs function as hash tables. As a result, EMBs display interesting phenomena, such as the birthday paradox, leaving EMBs severely under-utilized. RecShard determines an optimal EMB sharding strategy for a set of EMBs based on training data distributions and model characteristics, along with the bandwidth characteristics of the underlying tiered memory hierarchy. In doing so, RecShard achieves over 6 times higher EMB training throughput on average for capacity constrained DLRMs. The throughput increase comes from improved EMB load balance by over 12 times and from the reduced access to the slower memory by over 87 times. @InProceedings{ASPLOS22p344, author = {Geet Sethi and Bilge Acun and Niket Agarwal and Christos Kozyrakis and Caroline Trippel and Carole-Jean Wu}, title = {RecShard: Statistical Feature-Based Memory Optimization for Industry-Scale Neural Recommendation}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {344--358}, doi = {10.1145/3503222.3507777}, year = {2022}, } Publisher's Version
	Shahbaz, Muhammad	ASPLOS '22: "Taurus: A Data Plane Architecture ..." Taurus: A Data Plane Architecture for Per-Packet ML Tushar Swamy, Alexander Rucker, Muhammad Shahbaz, Ishan Gaur, and Kunle Olukotun (Stanford University, USA; Purdue University, USA) Emerging applications---cloud computing, the internet of things, and augmented/virtual reality---demand responsive, secure, and scalable datacenter networks. These networks currently implement simple, per-packet, data-plane heuristics (e.g., ECMP and sketches) under a slow, millisecond-latency control plane that runs data-driven performance and security policies. However, to meet applications' service-level objectives (SLOs) in a modern data center, networks must bridge the gap between line-rate, per-packet execution and complex decision making. In this work, we present the design and implementation of Taurus, a data plane for line-rate inference. Taurus adds custom hardware based on a flexible, parallel-patterns (MapReduce) abstraction to programmable network devices, such as switches and NICs; this new hardware uses pipelined SIMD parallelism to enable per-packet MapReduce operations (e.g., inference). Our evaluation of a Taurus switch ASIC---supporting several real-world models---shows that Taurus operates orders of magnitude faster than a server-based control plane while increasing area by 3.8% and latency for line-rate ML models by up to 221 ns. Furthermore, our Taurus FPGA prototype achieves full model accuracy and detects two orders of magnitude more events than a state-of-the-art control-plane anomaly-detection system. @InProceedings{ASPLOS22p1099, author = {Tushar Swamy and Alexander Rucker and Muhammad Shahbaz and Ishan Gaur and Kunle Olukotun}, title = {Taurus: A Data Plane Architecture for Per-Packet ML}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1099--1114}, doi = {10.1145/3503222.3507726}, year = {2022}, } Publisher's Version Published Artifact Info Artifacts Available
	Shalom, Gal	ASPLOS '22: "FlexDriver: A Network Driver ..." FlexDriver: A Network Driver for Your Accelerator Haggai Eran, Maxim Fudim, Gabi Malka, Gal Shalom, Noam Cohen, Amit Hermony, Dotan Levi, Liran Liss, and Mark Silberstein (NVIDIA, Israel; Technion, Israel) We propose a new system design for connecting hardware and FPGA accelerators to the network, allowing the accelerator to directly control commodity Network Interface Cards (NICs) without using the CPU. This enables us to solve the key challenge of leveraging existing NIC hardware offloads such as virtualization, tunneling, and RDMA for accelerator networking. Our approach supports a diverse set of use cases, from direct network access for disaggregated accelerators to inline-acceleration of the network stack, all without the complex networking logic in the accelerator. To demonstrate the feasibility of this approach, we build FlexDriver (FLD), an on-accelerator hardware module that implements a NIC data-plane driver. Our main technical contribution is a mechanism that compresses the NIC control structures by two orders of magnitude, allowing FLD to achieve high networking scalability with low die area cost and no bandwidth interference with the accelerator logic. The prototype for NVIDIA Innova-2 FPGA SmartNICs showcases our design’s utility for three different accelerators: a disaggregated LTE cipher, an IP-defragmentation inline accelerator, and an IoT cryptographic-token authentication offload. These accelerators reach 25 Gbps line rate and leverage the NIC for RDMA processing, VXLAN tunneling, and traffic shaping without CPU involvement. @InProceedings{ASPLOS22p1115, author = {Haggai Eran and Maxim Fudim and Gabi Malka and Gal Shalom and Noam Cohen and Amit Hermony and Dotan Levi and Liran Liss and Mark Silberstein}, title = {FlexDriver: A Network Driver for Your Accelerator}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1115--1129}, doi = {10.1145/3503222.3507776}, year = {2022}, } Publisher's Version
	Shan, Yizhou	ASPLOS '22: "Clio: A Hardware-Software ..." Clio: A Hardware-Software Co-Designed Disaggregated Memory System Zhiyuan Guo, Yizhou Shan, Xuhao Luo, Yutong Huang, and Yiying Zhang (University of California at San Diego, USA) Memory disaggregation has attracted great attention recently because of its benefits in efficient memory utilization and ease of management. So far, memory disaggregation research has all taken one of two approaches: building/emulating memory nodes using regular servers or building them using raw memory devices with no processing power. The former incurs higher monetary cost and faces tail latency and scalability limitations, while the latter introduces performance, security, and management problems. Server-based memory nodes and memory nodes with no processing power are two extreme approaches. We seek a sweet spot in the middle by proposing a hardware-based memory disaggregation solution that has the right amount of processing power at memory nodes. Furthermore, we take a clean-slate approach by starting from the requirements of memory disaggregation and designing a memory-disaggregation-native system. We built Clio, a disaggregated memory system that virtualizes, protects, and manages disaggregated memory at hardware-based memory nodes. The Clio hardware includes a new virtual memory system, a customized network system, and a framework for computation offloading. In building Clio, we not only co-design OS functionalities, hardware architecture, and the network system, but also co-design compute nodes and memory nodes. Our FPGA prototype of Clio demonstrates that each memory node can achieve 100 Gbps throughput and an end-to-end latency of 2.5 µ s at median and 3.2 µ s at the 99th percentile. Clio also scales much better and has orders of magnitude lower tail latency than RDMA. It has 1.1× to 3.4× energy saving compared to CPU-based and SmartNIC-based disaggregated memory systems and is 2.7× faster than software-based SmartNIC solutions. @InProceedings{ASPLOS22p417, author = {Zhiyuan Guo and Yizhou Shan and Xuhao Luo and Yutong Huang and Yiying Zhang}, title = {Clio: A Hardware-Software Co-Designed Disaggregated Memory System}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {417--433}, doi = {10.1145/3503222.3507762}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional
	Sharma, Bikash	ASPLOS '22: "TMO: Transparent Memory Offloading ..." TMO: Transparent Memory Offloading in Datacenters Johannes Weiner, Niket Agarwal, Dan Schatzberg, Leon Yang, Hao Wang, Blaise Sanouillet, Bikash Sharma, Tejun Heo, Mayank Jain, Chunqiang Tang, and Dimitrios Skarlatos (Meta, USA; Carnegie Mellon University, USA) The unrelenting growth of the memory needs of emerging datacenter applications, along with ever increasing cost and volatility of DRAM prices, has led to DRAM being a major infrastructure expense. Alternative technologies, such as NVMe SSDs and upcoming NVM devices, offer higher capacity than DRAM at a fraction of the cost and power. One promising approach is to transparently offload colder memory to cheaper memory technologies via kernel or hypervisor techniques. The key challenge, however, is to develop a datacenter-scale solution that is robust in dealing with diverse workloads and large performance variance of different offload devices such as compressed memory, SSD, and NVM. This paper presents TMO, Meta’s transparent memory offloading solution for heterogeneous datacenter environments. TMO introduces a new Linux kernel mechanism that directly measures in realtime the lost work due to resource shortage across CPU, memory, and I/O. Guided by this information and without any prior application knowledge, TMO automatically adjusts how much memory to offload to heterogeneous devices (e.g., compressed memory or SSD) according to the device’s performance characteristics and the application’s sensitivity to memory-access slowdown. TMO holistically identifies offloading opportunities from not only the application containers but also the sidecar containers that provide infrastructure-level functions. To maximize memory savings, TMO targets both anonymous memory and file cache, and balances the swap-in rate of anonymous memory and the reload rate of file pages that were recently evicted from the file cache. TMO has been running in production for more than a year, and has saved between 20-32% of the total memory across millions of servers in our large datacenter fleet. We have successfully upstreamed TMO into the Linux kernel. @InProceedings{ASPLOS22p609, author = {Johannes Weiner and Niket Agarwal and Dan Schatzberg and Leon Yang and Hao Wang and Blaise Sanouillet and Bikash Sharma and Tejun Heo and Mayank Jain and Chunqiang Tang and Dimitrios Skarlatos}, title = {TMO: Transparent Memory Offloading in Datacenters}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {609--621}, doi = {10.1145/3503222.3507731}, year = {2022}, } Publisher's Version
	Sharma, Prateek	ASPLOS '22: "Memory-Harvesting VMs in Cloud ..." Memory-Harvesting VMs in Cloud Platforms Alexander Fuerst, Stanko Novaković, Íñigo Goiri, Gohar Irfan Chaudhry, Prateek Sharma, Kapil Arya, Kevin Broas, Eugene Bak, Mehmet Iyigun, and Ricardo Bianchini (Indiana University, USA; Microsoft Research, USA; Microsoft Azure, USA) loud platforms monetize their spare capacity by renting “Spot” virtual machines (VMs) that can be evicted in favor of higher-priority VMs. Recent work has shown that resource-harvesting VMs are more effective at exploiting spare capacity than Spot VMs, while also reducing the number of evictions. However, the prior work focused on harvesting CPU cores while keeping memory size fixed. This wastes a substantial monetization opportunity and may even limit the ability of harvesting VMs to leverage spare cores. Thus, in this paper, we explore memory harvesting and its challenges in real cloud platforms, namely its impact on VM creation time, NUMA spanning, and page fragmentation. We start by characterizing the amount and dynamics of the spare memory in Azure. We then design and implement memory-harvesting VMs (MHVMs), introducing new techniques for memory buffering, batching, and pre-reclamation. To demonstrate the use of MHVMs, we also extend a popular cluster scheduling framework (Hadoop) and a FaaS platform to adapt to them. Our main results show that (1) there is plenty of scope for memory harvesting in real platforms; (2) MHVMs are effective at mitigating the negative impacts of harvesting; and (3) our extensions of Hadoop and FaaS successfully hide the MHVMs’ varying memory size from the users’ data-processing jobs and functions. We conclude that memory harvesting has great potential for practical deployment and users can save up to 93% of their costs when running workloads on MHVMs. @InProceedings{ASPLOS22p583, author = {Alexander Fuerst and Stanko Novaković and Íñigo Goiri and Gohar Irfan Chaudhry and Prateek Sharma and Kapil Arya and Kevin Broas and Eugene Bak and Mehmet Iyigun and Ricardo Bianchini}, title = {Memory-Harvesting VMs in Cloud Platforms}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {583--594}, doi = {10.1145/3503222.3507725}, year = {2022}, } Publisher's Version
	Shen, Tianxiang	ASPLOS '22: "NASPipe: High Performance ..." NASPipe: High Performance and Reproducible Pipeline Parallel Supernet Training via Causal Synchronous Parallelism Shixiong Zhao, Fanxin Li, Xusheng Chen, Tianxiang Shen, Li Chen, Sen Wang, Nicholas Zhang, Cheng Li, and Heming Cui (University of Hong Kong, China; Huawei Technologies, China; University of Science and Technology of China, China) Supernet training, a prevalent and important paradigm in Neural Architecture Search, embeds the whole DNN architecture search space into one monolithic supernet, iteratively activates a subset of the supernet (i.e., a subnet) for fitting each batch of data, and searches a high-quality subnet which meets specific requirements. Although training subnets in parallel on multiple GPUs is desirable for acceleration, there inherently exists a race hazard that concurrent subnets may access the same DNN layers. Existing systems support neither efficiently parallelizing subnets’ training executions, nor resolving the race hazard deterministically, leading to unreproducible training procedures and potentiallly non-trivial accuracy loss. We present NASPipe, the first high-performance and reproducible distributed supernet training system via causal synchronous parallel (CSP) pipeline scheduling abstraction: NASPipe partitions a supernet across GPUs and concurrently executes multiple generated sub-tasks (subnets) in a pipelined manner; meanwhile, it oversees the correlations between the subnets and deterministically resolves any causal dependency caused by subnets’ layer sharing. To obtain high performance, NASPipe’s CSP scheduler exploits the fact that the larger a supernet spans, the fewer dependencies manifest between chronologically close subnets; therefore, it aggressively schedules the subnets with larger chronological orders into execution, only if they are not causally dependent on unfinished precedent subnets. Moreover, to relieve the excessive GPU memory burden for holding the whole supernet’s parameters, NASPipe uses a context switch technique that stashes the whole supernet in CPU memory, precisely predicts the subnets’ schedule, and pre-fetches/evicts a subnet before/after its execution. The evaluation shows that NASPipe is the only system that retains supernet training reproducibility, while achieving a comparable and even higher performance (up to 7.8X) compared to three recent pipeline training systems (e.g., GPipe). @InProceedings{ASPLOS22p374, author = {Shixiong Zhao and Fanxin Li and Xusheng Chen and Tianxiang Shen and Li Chen and Sen Wang and Nicholas Zhang and Cheng Li and Heming Cui}, title = {NASPipe: High Performance and Reproducible Pipeline Parallel Supernet Training via Causal Synchronous Parallelism}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {374--387}, doi = {10.1145/3503222.3507735}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Shi, Yunong	ASPLOS '22: "Paulihedral: A Generalized ..." Paulihedral: A Generalized Block-Wise Compiler Optimization Framework for Quantum Simulation Kernels Gushu Li, Anbang Wu, Yunong Shi, Ali Javadi-Abhari, Yufei Ding, and Yuan Xie (University of California at Santa Barbara, USA; Amazon, USA; IBM, USA) The quantum simulation kernel is an important subroutine appearing as a very long gate sequence in many quantum programs. In this paper, we propose Paulihedral, a block-wise compiler framework that can deeply optimize this subroutine by exploiting high-level program structure and optimization opportunities. Paulihedral first employs a new Pauli intermediate representation that can maintain the high-level semantics and constraints in quantum simulation kernels. This naturally enables new large-scale optimizations that are hard to implement at the low gate-level. In particular, we propose two technology-independent instruction scheduling passes, and two technology-dependent code optimization passes which reconcile the circuit synthesis, gate cancellation, and qubit mapping stages of the compiler. Experimental results show that Paulihedral can outperform state-of-the-art compiler infrastructures in a wide-range of applications on both near-term superconducting quantum processors and future fault-tolerant quantum computers. @InProceedings{ASPLOS22p554, author = {Gushu Li and Anbang Wu and Yunong Shi and Ali Javadi-Abhari and Yufei Ding and Yuan Xie}, title = {Paulihedral: A Generalized Block-Wise Compiler Optimization Framework for Quantum Simulation Kernels}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {554--569}, doi = {10.1145/3503222.3507715}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Shoshitaishvili, Yan	ASPLOS '22: "ViK: Practical Mitigation ..." ViK: Practical Mitigation of Temporal Memory Safety Violations through Object ID Inspection Haehyun Cho, Jinbum Park, Adam Oest, Tiffany Bao, Ruoyu Wang, Yan Shoshitaishvili, Adam Doupé, and Gail-Joon Ahn (Soongsil University, South Korea; Samsung Research, South Korea; PayPal, USA; Arizona State University, USA; Samsung Research, USA) Temporal memory safety violations, such as use-after-free (UAF) vulnerabilities, are a critical security issue for software written in memory-unsafe languages such as C and C++. In this paper, we introduce ViK, a novel, lightweight, and widely applicable runtime defense that can protect both operating system (OS) kernels and user-space applications against temporal memory safety violations. ViK performs object ID inspection, where it assigns a random identifier to every allocated object and stores the identifier in the unused bits of the corresponding pointer. When the pointer is used, ViK inspects the value of a pointer before dereferencing, ensuring that the pointer still references the original object. To the best of our knowledge, this is the first mitigation against temporal memory safety violations that scales to OS kernels. We evaluated the software prototype of ViK on Android and Linux kernels and observed runtime overhead of around 20%. Also, we evaluated a hardware-assisted prototype of ViK on Android kernel, where the runtime overhead was as low as 2%. @InProceedings{ASPLOS22p271, author = {Haehyun Cho and Jinbum Park and Adam Oest and Tiffany Bao and Ruoyu Wang and Yan Shoshitaishvili and Adam Doupé and Gail-Joon Ahn}, title = {ViK: Practical Mitigation of Temporal Memory Safety Violations through Object ID Inspection}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {271--284}, doi = {10.1145/3503222.3507780}, year = {2022}, } Publisher's Version Artifacts Functional
	Silberstein, Mark	ASPLOS '22: "FlexDriver: A Network Driver ..." FlexDriver: A Network Driver for Your Accelerator Haggai Eran, Maxim Fudim, Gabi Malka, Gal Shalom, Noam Cohen, Amit Hermony, Dotan Levi, Liran Liss, and Mark Silberstein (NVIDIA, Israel; Technion, Israel) We propose a new system design for connecting hardware and FPGA accelerators to the network, allowing the accelerator to directly control commodity Network Interface Cards (NICs) without using the CPU. This enables us to solve the key challenge of leveraging existing NIC hardware offloads such as virtualization, tunneling, and RDMA for accelerator networking. Our approach supports a diverse set of use cases, from direct network access for disaggregated accelerators to inline-acceleration of the network stack, all without the complex networking logic in the accelerator. To demonstrate the feasibility of this approach, we build FlexDriver (FLD), an on-accelerator hardware module that implements a NIC data-plane driver. Our main technical contribution is a mechanism that compresses the NIC control structures by two orders of magnitude, allowing FLD to achieve high networking scalability with low die area cost and no bandwidth interference with the accelerator logic. The prototype for NVIDIA Innova-2 FPGA SmartNICs showcases our design’s utility for three different accelerators: a disaggregated LTE cipher, an IP-defragmentation inline accelerator, and an IoT cryptographic-token authentication offload. These accelerators reach 25 Gbps line rate and leverage the NIC for RDMA processing, VXLAN tunneling, and traffic shaping without CPU involvement. @InProceedings{ASPLOS22p1115, author = {Haggai Eran and Maxim Fudim and Gabi Malka and Gal Shalom and Noam Cohen and Amit Hermony and Dotan Levi and Liran Liss and Mark Silberstein}, title = {FlexDriver: A Network Driver for Your Accelerator}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1115--1129}, doi = {10.1145/3503222.3507776}, year = {2022}, } Publisher's Version ASPLOS '22: "Revizor: Testing Black-Box ..." Revizor: Testing Black-Box CPUs against Speculation Contracts Oleksii Oleksenko, Christof Fetzer, Boris Köpf, and Mark Silberstein (TU Dresden, Germany; Microsoft Research, UK; Technion, Israel) Speculative vulnerabilities such as Spectre and Meltdown expose speculative execution state that can be exploited to leak information across security domains via side-channels. Such vulnerabilities often stay undetected for a long time as we lack the tools for systematic testing of CPUs to find them. In this paper, we propose an approach to automatically detect microarchitectural information leakage in commercial black-box CPUs. We build on speculation contracts, which we employ to specify the permitted side effects of program execution on the CPU's microarchitectural state. We propose a Model-based Relational Testing (MRT) technique to empirically assess the CPU compliance with these specifications. We implement MRT in a testing framework called Revizor, and showcase its effectiveness on real Intel x86 CPUs. Revizor automatically detects violations of a rich set of contracts, or indicates their absence. A highlight of our findings is that Revizor managed to automatically surface Spectre, MDS, and LVI, as well as several previously unknown variants. @InProceedings{ASPLOS22p226, author = {Oleksii Oleksenko and Christof Fetzer and Boris Köpf and Mark Silberstein}, title = {Revizor: Testing Black-Box CPUs against Speculation Contracts}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {226--239}, doi = {10.1145/3503222.3507729}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Skarlatos, Dimitrios	ASPLOS '22: "IOCost: Block IO Control for ..." IOCost: Block IO Control for Containers in Datacenters Tejun Heo, Dan Schatzberg, Andrew Newell, Song Liu, Saravanan Dhakshinamurthy, Iyswarya Narayanan, Josef Bacik, Chris Mason, Chunqiang Tang, and Dimitrios Skarlatos (Meta, USA; Carnegie Mellon University, USA) Resource isolation is a fundamental requirement in datacenter environments. However, our production experience in Meta’s large-scale datacenters shows that existing IO control mechanisms for block storage are inadequate in containerized environments. IO control needs to provide proportional resources to containers while taking into account the hardware heterogeneity of storage devices and the idiosyncrasies of the workloads deployed in datacenters. The speed of modern SSDs requires IO control to execute with low-overheads. Furthermore, IO control should strive for work conservation, take into account the interactions with the memory management subsystem, and avoid priority inversions that lead to isolation failures. To address these challenges, this paper presents IOCost, an IO control solution that is designed for containerized environments and provides scalable, work-conserving, and low-overhead IO control for heterogeneous storage devices and diverse workloads in datacenters. IOCost performs offline profiling to build a device model and uses it to estimate device occupancy of each IO request. To minimize runtime overhead, it separates IO control into a fast per-IO issue path and a slower periodic planning path. A novel work-conserving budget donation algorithm enables containers to dynamically share unused budget. We have deployed IOCost across the entirety of Meta’s datacenters comprised of millions of ma- chines, upstreamed IOCost to the Linux kernel, and open-sourced our device-profiling tools. IOCost has been running in production for two years, providing IO control for Meta’s fleet. We describe the design of IOCost and share our experience deploying it at scale. @InProceedings{ASPLOS22p595, author = {Tejun Heo and Dan Schatzberg and Andrew Newell and Song Liu and Saravanan Dhakshinamurthy and Iyswarya Narayanan and Josef Bacik and Chris Mason and Chunqiang Tang and Dimitrios Skarlatos}, title = {IOCost: Block IO Control for Containers in Datacenters}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {595--608}, doi = {10.1145/3503222.3507727}, year = {2022}, } Publisher's Version ASPLOS '22: "TMO: Transparent Memory Offloading ..." TMO: Transparent Memory Offloading in Datacenters Johannes Weiner, Niket Agarwal, Dan Schatzberg, Leon Yang, Hao Wang, Blaise Sanouillet, Bikash Sharma, Tejun Heo, Mayank Jain, Chunqiang Tang, and Dimitrios Skarlatos (Meta, USA; Carnegie Mellon University, USA) The unrelenting growth of the memory needs of emerging datacenter applications, along with ever increasing cost and volatility of DRAM prices, has led to DRAM being a major infrastructure expense. Alternative technologies, such as NVMe SSDs and upcoming NVM devices, offer higher capacity than DRAM at a fraction of the cost and power. One promising approach is to transparently offload colder memory to cheaper memory technologies via kernel or hypervisor techniques. The key challenge, however, is to develop a datacenter-scale solution that is robust in dealing with diverse workloads and large performance variance of different offload devices such as compressed memory, SSD, and NVM. This paper presents TMO, Meta’s transparent memory offloading solution for heterogeneous datacenter environments. TMO introduces a new Linux kernel mechanism that directly measures in realtime the lost work due to resource shortage across CPU, memory, and I/O. Guided by this information and without any prior application knowledge, TMO automatically adjusts how much memory to offload to heterogeneous devices (e.g., compressed memory or SSD) according to the device’s performance characteristics and the application’s sensitivity to memory-access slowdown. TMO holistically identifies offloading opportunities from not only the application containers but also the sidecar containers that provide infrastructure-level functions. To maximize memory savings, TMO targets both anonymous memory and file cache, and balances the swap-in rate of anonymous memory and the reload rate of file pages that were recently evicted from the file cache. TMO has been running in production for more than a year, and has saved between 20-32% of the total memory across millions of servers in our large datacenter fleet. We have successfully upstreamed TMO into the Linux kernel. @InProceedings{ASPLOS22p609, author = {Johannes Weiner and Niket Agarwal and Dan Schatzberg and Leon Yang and Hao Wang and Blaise Sanouillet and Bikash Sharma and Tejun Heo and Mayank Jain and Chunqiang Tang and Dimitrios Skarlatos}, title = {TMO: Transparent Memory Offloading in Datacenters}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {609--621}, doi = {10.1145/3503222.3507731}, year = {2022}, } Publisher's Version ASPLOS '22: "Parallel Virtualized Memory ..." Parallel Virtualized Memory Translation with Nested Elastic Cuckoo Page Tables Jovan Stojkovic, Dimitrios Skarlatos, Apostolos Kokolis, Tianyin Xu, and Josep Torrellas (University of Illinois at Urbana-Champaign, USA; Carnegie Mellon University, USA) A major reason why nested or virtualized address translations are slow is because current systems organize page tables in a multi-level tree that is accessed in a sequential manner. A nested translation may potentially require up to twenty-four sequential memory accesses. To address this problem, this paper presents the first page table design that supports parallel nested address translation. The design is based on using hashed page tables (HPTs) for both guest and host. However, directly extending a native HPT design to a nested environment leads to minor gains. Instead, our design solves a new set of challenges that appear in nested environments. Our scheme eliminates all but three of the potentially twenty-four sequential steps of a nested translation—while judiciously limiting the number of parallel memory accesses issued to avoid over-consuming cache bandwidth. As a result, compared to conventional nested radix tables, our design speeds-up the execution of a set of applications by an average of 1.19x (for 4KB pages) and 1.24x (when huge pages are used). In addition, we also show a migration path from current nested radix page tables to our design. @InProceedings{ASPLOS22p84, author = {Jovan Stojkovic and Dimitrios Skarlatos and Apostolos Kokolis and Tianyin Xu and Josep Torrellas}, title = {Parallel Virtualized Memory Translation with Nested Elastic Cuckoo Page Tables}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {84--97}, doi = {10.1145/3503222.3507720}, year = {2022}, } Publisher's Version
	Song, Linhai	ASPLOS '22: "Who Goes First? Detecting ..." Who Goes First? Detecting Go Concurrency Bugs via Message Reordering Ziheng Liu, Shihao Xia, Yu Liang, Linhai Song, and Hong Hu (Pennsylvania State University, USA) Go is a young programming language invented to build safe and efficient concurrent programs. It provides goroutines as lightweight threads and channels for inter-goroutine communication. Programmers are encouraged to explicitly pass messages through channels to connect goroutines, with the purpose of reducing the chance of making programming mistakes and introducing concurrency bugs. Go is one of the most beloved programming languages and has already been used to build many critical infrastructure software systems in the data-center environment. However, a recent study shows that channel-related concurrency bugs are still common in Go programs, severely hurting the reliability of the programs. This paper presents GFuzz, a dynamic detector that can effectively pinpoint channel-related concurrency bugs by mutating the processing orders of concurrent messages. We build GFuzz in three steps. We first adopt an effective approach to identify concurrent messages and transform a program to process those messages in any given order. We then take a fuzzing approach to generate new processing orders by mutating exercised ones and rely on execution feedback to prioritize orders close to triggering bugs. Finally, we design a runtime sanitizer to capture triggered bugs that are missed by the Go runtime. We evaluate GFuzz on seven popular Go software systems, including Docker, Kubernetes, and gRPC. GFuzz finds 184 previously unknown bugs and reports a negligible number of false positives. Programmers have already confirmed 124 reports as real bugs and fixed 67 of them based on our reporting. A careful inspection of the detected concurrency bugs from gRPC shows the effectiveness of each component of GFuzz and confirms the components' rationality. @InProceedings{ASPLOS22p888, author = {Ziheng Liu and Shihao Xia and Yu Liang and Linhai Song and Hong Hu}, title = {Who Goes First? Detecting Go Concurrency Bugs via Message Reordering}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {888--902}, doi = {10.1145/3503222.3507753}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional
	Song, Shuaiwen Leon	ASPLOS '22: "AStitch: Enabling a New Multi-dimensional ..." AStitch: Enabling a New Multi-dimensional Optimization Space for Memory-Intensive ML Training and Inference on Modern SIMT Architectures Zhen Zheng, Xuanda Yang, Pengzhan Zhao, Guoping Long, Kai Zhu, Feiwen Zhu, Wenyi Zhao, Xiaoyong Liu, Jun Yang, Jidong Zhai, Shuaiwen Leon Song, and Wei Lin (Alibaba Group, China; Tsinghua University, China; University of Sydney, Australia) This work reveals that memory-intensive computation is a rising performance-critical factor in recent machine learning models. Due to a unique set of new challenges, existing ML optimizing compilers cannot perform efficient fusion under complex two-level dependencies combined with just-in-time demand. They face the dilemma of either performing costly fusion due to heavy redundant computation, or skipping fusion which results in massive number of kernels. Furthermore, they often suffer from low parallelism due to the lack of support for real-world production workloads with irregular tensor shapes. To address these rising challenges, we propose AStitch, a machine learning optimizing compiler that opens a new multi-dimensional optimization space for memory-intensive ML computations. It systematically abstracts four operator-stitching schemes while considering multi-dimensional optimization objectives, tackles complex computation graph dependencies with novel hierarchical data reuse, and efficiently processes various tensor shapes via adaptive thread mapping. Finally, AStitch provides just-in-time support incorporating our proposed optimizations for both ML training and inference. Although AStitch serves as a stand-alone compiler engine that is portable to any version of TensorFlow, its basic ideas can be generally applied to other ML frameworks and optimization compilers. Experimental results show that AStitch can achieve an average of 1.84x speedup (up to 2.73x) over the state-of-the-art Google's XLA solution across five production workloads. We also deploy AStitch onto a production cluster for ML workloads with thousands of GPUs. The system has been in operation for more than 10 months and saves about 20,000 GPU hours for 70,000 tasks per week. @InProceedings{ASPLOS22p359, author = {Zhen Zheng and Xuanda Yang and Pengzhan Zhao and Guoping Long and Kai Zhu and Feiwen Zhu and Wenyi Zhao and Xiaoyong Liu and Jun Yang and Jidong Zhai and Shuaiwen Leon Song and Wei Lin}, title = {AStitch: Enabling a New Multi-dimensional Optimization Space for Memory-Intensive ML Training and Inference on Modern SIMT Architectures}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {359--373}, doi = {10.1145/3503222.3507723}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Songhori, Ebrahim	ASPLOS '22: "A Full-Stack Search Technique ..." A Full-Stack Search Technique for Domain Optimized Deep Learning Accelerators Dan Zhang, Safeen Huda, Ebrahim Songhori, Kartik Prabhu, Quoc Le, Anna Goldie, and Azalia Mirhoseini (Google Brain, USA; Google, USA; Stanford University, USA) The rapidly-changing deep learning landscape presents a unique opportunity for building inference accelerators optimized for specific datacenter-scale workloads. We propose Full-stack Accelerator Search Technique (FAST), a hardware accelerator search framework that defines a broad optimization environment covering key design decisions within the hardware-software stack, including hardware datapath, software scheduling, and compiler passes such as operation fusion and tensor padding. In this paper, we analyze bottlenecks in state-of-the-art vision and natural language processing (NLP) models, including EfficientNet and BERT, and use FAST to design accelerators capable of addressing these bottlenecks. FAST-generated accelerators optimized for single workloads improve Perf/TDP by 3.7× on average across all benchmarks compared to TPU-v3. A FAST-generated accelerator optimized for serving a suite of workloads improves Perf/TDP by 2.4× on average compared to TPU-v3. Our return on investment analysis shows that FAST-generated accelerators can potentially be practical for moderate-sized datacenter deployments. @InProceedings{ASPLOS22p27, author = {Dan Zhang and Safeen Huda and Ebrahim Songhori and Kartik Prabhu and Quoc Le and Anna Goldie and Azalia Mirhoseini}, title = {A Full-Stack Search Technique for Domain Optimized Deep Learning Accelerators}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {27--42}, doi = {10.1145/3503222.3507767}, year = {2022}, } Publisher's Version
	Stojkovic, Jovan	ASPLOS '22: "Parallel Virtualized Memory ..." Parallel Virtualized Memory Translation with Nested Elastic Cuckoo Page Tables Jovan Stojkovic, Dimitrios Skarlatos, Apostolos Kokolis, Tianyin Xu, and Josep Torrellas (University of Illinois at Urbana-Champaign, USA; Carnegie Mellon University, USA) A major reason why nested or virtualized address translations are slow is because current systems organize page tables in a multi-level tree that is accessed in a sequential manner. A nested translation may potentially require up to twenty-four sequential memory accesses. To address this problem, this paper presents the first page table design that supports parallel nested address translation. The design is based on using hashed page tables (HPTs) for both guest and host. However, directly extending a native HPT design to a nested environment leads to minor gains. Instead, our design solves a new set of challenges that appear in nested environments. Our scheme eliminates all but three of the potentially twenty-four sequential steps of a nested translation—while judiciously limiting the number of parallel memory accesses issued to avoid over-consuming cache bandwidth. As a result, compared to conventional nested radix tables, our design speeds-up the execution of a set of applications by an average of 1.19x (for 4KB pages) and 1.24x (when huge pages are used). In addition, we also show a migration path from current nested radix page tables to our design. @InProceedings{ASPLOS22p84, author = {Jovan Stojkovic and Dimitrios Skarlatos and Apostolos Kokolis and Tianyin Xu and Josep Torrellas}, title = {Parallel Virtualized Memory Translation with Nested Elastic Cuckoo Page Tables}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {84--97}, doi = {10.1145/3503222.3507720}, year = {2022}, } Publisher's Version
	Stone, Cathlyn	ASPLOS '22: "Adelie: Continuous Address ..." Adelie: Continuous Address Space Layout Re-randomization for Linux Drivers Ruslan Nikolaev, Hassan Nadeem, Cathlyn Stone, and Binoy Ravindran (Pennsylvania State University, USA; Virginia Tech, USA) While address space layout randomization (ASLR) has been extensively studied for user-space programs, the corresponding OS kernel's KASLR support remains very limited, making the kernel vulnerable to just-in-time (JIT) return-oriented programming (ROP) attacks. Furthermore, commodity OSs such as Linux restrict their KASLR range to 32 bits due to architectural constraints (e.g., x86-64 only supports 32-bit immediate operands for most instructions), which makes them vulnerable to even unsophisticated brute-force ROP attacks due to low entropy. Most in-kernel pointers remain static, exacerbating the problem when pointers are leaked. Adelie, our kernel defense mechanism, overcomes KASLR limitations, increases KASLR entropy, and makes successful ROP attacks on the Linux kernel much harder to achieve. First, Adelie enables the position-independent code (PIC) model so that the kernel and its modules can be placed anywhere in the 64-bit virtual address space, at any distance apart from each other. Second, Adelie implements stack re-randomization and address encryption on modules. Finally, Adelie enables efficient continuous KASLR for modules by using the PIC model to make it (almost) impossible to inject ROP gadgets through these modules regardless of gadget's origin. Since device drivers (typically compiled as modules) are often developed by third parties and are typically less tested than core OS parts, they are also often more vulnerable. By fully re-randomizing device drivers, the last two contributions together prevent most JIT ROP attacks since vulnerable modules are very likely to be a starting point of an attack. Furthermore, some OS instances in virtualized environments are specifically designated to run device drivers, where drivers are the primary target of JIT ROP attacks. Using a GCC plugin that we developed, we automatically modify different kinds of kernel modules. Since the prior art tackles only user-space programs, we solve many challenges unique to the kernel code. Our evaluation shows high efficiency of Adelie's approach: the overhead of the PIC model is completely negligible and re-randomization cost remains reasonable for typical use cases. @InProceedings{ASPLOS22p483, author = {Ruslan Nikolaev and Hassan Nadeem and Cathlyn Stone and Binoy Ravindran}, title = {Adelie: Continuous Address Space Layout Re-randomization for Linux Drivers}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {483--498}, doi = {10.1145/3503222.3507779}, year = {2022}, } Publisher's Version Published Artifact Info Artifacts Available Artifacts Functional Results Reproduced
	Su, Zhendong	ASPLOS '22: "Finding Missed Optimizations ..." Finding Missed Optimizations through the Lens of Dead Code Elimination Theodoros Theodoridis, Manuel Rigger, and Zhendong Su (ETH Zurich, Switzerland) Compilers are foundational software development tools and incorporate increasingly sophisticated optimizations. Due to their complexity, it is difficult to systematically identify opportunities for improving them. Indeed, the automatic discovery of missed optimizations has been an important and significant challenge. The few existing approaches either cannot accurately pinpoint missed optimizations or target only specific analyses. This paper tackles this challenge by introducing a novel, effective approach that --- in a simple and general manner --- automatically identifies a wide range of missed optimizations. Our core insight is to leverage dead code elimination (DCE) to both analyze how well compilers optimize code and identify missed optimizations: (1) insert "optimization markers" in the basic blocks of a given program, (2) compute the program's live/dead basic blocks using the "optimization markers", and (3) identify missed optimizations from how well compilers eliminate dead blocks. We essentially exploit that, since DCE heavily depends on the rest of the optimization pipeline, through the lens of DCE, one can systematically quantify how well compilers optimize code. We conduct an extensive analysis of GCC and LLVM using our approach, which (1) provides quantitative and qualitative insights regarding their optimization capabilities, and (2) uncovers a diverse set of missed optimizations. Our results also lead to 84 bug reports for GCC and LLVM, of which 62 have already been confirmed or fixed, demonstrating our work's strong practical utility. We expect that the simplicity and generality of our approach will make it widely applicable for understanding compiler performance and finding missed optimizations. This work opens and initiates this promising direction. @InProceedings{ASPLOS22p697, author = {Theodoros Theodoridis and Manuel Rigger and Zhendong Su}, title = {Finding Missed Optimizations through the Lens of Dead Code Elimination}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {697--709}, doi = {10.1145/3503222.3507764}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced ASPLOS '22: "Understanding and Exploiting ..." Understanding and Exploiting Optimal Function Inlining Theodoros Theodoridis, Tobias Grosser, and Zhendong Su (ETH Zurich, Switzerland; University of Edinburgh, UK) Inlining is a core transformation in optimizing compilers. It replaces a function call (call site) with the body of the called function (callee). It helps reduce function call overhead and binary size, and more importantly, enables other optimizations. The problem of inlining has been extensively studied, but it is far from being solved; predicting which inlining decisions are beneficial is nontrivial due to interactions with the rest of the compiler pipeline. Previous work has mainly focused on designing heuristics for better inlining decisions and has not investigated optimal inlining, i.e., exhaustively finding the optimal inlining decisions. Optimal inlining is necessary for identifying and exploiting missed opportunities and evaluating the state of the art. This paper fills this gap through an extensive empirical analysis of optimal inlining using the SPEC2017 benchmark suite. Our novel formulation drastically reduces the inlining search space size (from 2349 down to 225) and allows us to exhaustively evaluate all inlining choices on 1,135 SPEC2017 files. We show a significant gap between the state-of-the-art strategy in LLVM and optimal inlining when optimizing for binary size, an important, deterministic metric independent of workload (in contrast to performance, another important metric). Inspired by our analysis, we introduce a simple, effective autotuning strategy for inlining that outperforms the state of the art by 7% on average (and up to 28%) on SPEC2017, 15% on the source code of LLVM itself, and 10% on the source code of SQLite. This work highlights the importance of exploring optimal inlining by providing new, actionable insight and an effective autotuning strategy that is of practical utility. @InProceedings{ASPLOS22p977, author = {Theodoros Theodoridis and Tobias Grosser and Zhendong Su}, title = {Understanding and Exploiting Optimal Function Inlining}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {977--989}, doi = {10.1145/3503222.3507744}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Suchy, Brian	ASPLOS '22: "CARAT CAKE: Replacing Paging ..." CARAT CAKE: Replacing Paging via Compiler/Kernel Cooperation Brian Suchy, Souradip Ghosh, Drew Kersnar, Siyuan Chai, Zhen Huang, Aaron Nelson, Michael Cuevas, Alex Bernat, Gaurav Chaudhary, Nikos Hardavellas, Simone Campanoni, and Peter Dinda (Northwestern University, USA) Virtual memory, specifically paging, is undergoing significant innovation due to being challenged by new demands from modern workloads. Recent work has demonstrated an alternative software only design that can result in simplified hardware requirements, even supporting purely physical addressing. While we have made the case for this Compiler- And Runtime-based Address Translation (CARAT) concept, its evaluation was based on a user-level prototype. We now report on incorporating CARAT into a kernel, forming Compiler- And Runtime-based Address Translation for CollAborative Kernel Environments (CARAT CAKE). In our implementation, a Linux-compatible x64 process abstraction can be based either on CARAT CAKE, or on a sophisticated paging implementation. Implementing CARAT CAKE involves kernel changes and compiler optimizations/transformations that must work on all code in the system, including kernel code. We evaluate CARAT CAKE in comparison with paging and find that CARAT CAKE is able to achieve the functionality of paging (protection, mapping, and movement properties) with minimal overhead. In turn, CARAT CAKE allows significant new benefits for systems including energy savings, larger L1 caches, and arbitrary granularity memory management. @InProceedings{ASPLOS22p98, author = {Brian Suchy and Souradip Ghosh and Drew Kersnar and Siyuan Chai and Zhen Huang and Aaron Nelson and Michael Cuevas and Alex Bernat and Gaurav Chaudhary and Nikos Hardavellas and Simone Campanoni and Peter Dinda}, title = {CARAT CAKE: Replacing Paging via Compiler/Kernel Cooperation}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {98--114}, doi = {10.1145/3503222.3507771}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available
	Sui, Yulei	ASPLOS '22: "Path-Sensitive and Alias-Aware ..." Path-Sensitive and Alias-Aware Typestate Analysis for Detecting OS Bugs Tuo Li, Jia-Ju Bai, Yulei Sui, and Shi-Min Hu (Tsinghua University, China; University of Technology Sydney, Australia) Operating system (OS) is the cornerstone for modern computer systems. It manages devices and provides fundamental service for user-level applications. Thus, detecting bugs in OSes is important to improve reliability and security of computer systems. Static typestate analysis is a common technique for detecting different types of bugs, but it is often inaccurate or unscalable for large-size OS code, due to imprecision of identifying alias relationships as well as high costs of typestate tracking and path-feasibility validation. In this paper, we present PATA, a novel path-sensitive and aliasaware typestate analysis framework to detect OS bugs. To improve the precision of identifying alias relationships in OS code, PATA performs a path-based alias analysis based on control-flow paths and access paths. With these alias relationships, PATA reduces the costs of typestate tracking and path-feasibility validation, to boost the efficiency of path-sensitive typestate analysis for bug detection. We have evaluated PATA on the Linux kernel and three popular IoT OSes (Zephyr, RIOT and TencentOS-tiny) to detect three common types of bugs (null-pointer dereferences, uninitialized variable accesses and memory leaks). PATA finds 574 real bugs with a false positive rate of 28%. 206 of these bugs have been confirmed by the developers of the four OSes.We also compare PATA to seven state-of-the-art static approaches (Cppcheck, Coccinelle, Smatch,CSA, Infer, Saber and SVF). PATA finds many real bugs missed by them, with a lower false positive rate. @InProceedings{ASPLOS22p859, author = {Tuo Li and Jia-Ju Bai and Yulei Sui and Shi-Min Hu}, title = {Path-Sensitive and Alias-Aware Typestate Analysis for Detecting OS Bugs}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {859--872}, doi = {10.1145/3503222.3507770}, year = {2022}, } Publisher's Version
	Sun, Fei	ASPLOS '22: "A One-for-All and O(V ..." A One-for-All and O(V log(V ))-Cost Solution for Parallel Merge Style Operations on Sorted Key-Value Arrays Bangyan Wang, Lei Deng, Fei Sun, Guohao Dai, Liu Liu, Yu Wang, and Yuan Xie (University of California at Santa Barbara, USA; Tsinghua University, China; Alibaba DAMO Academy, China) The processing of sorted key-value arrays using a “merge style operation (MSO)” is a very basic and important problem in domains like scientific computing, deep learning, database, graph analysis, sorting, set-operation etc. MSOs dominate the execution time in some important applications like SpGEMM and graph mining. For example, sparse vector addition as an MSO takes up to 98% execution time in SpGEMM in our experiment. For this reason, accelerating MSOs on CPU, GPU, and accelerators using parallel execution has been extensively studied but the solutions in prior work have three major limitations. (1) They treat different MSOs as isolated problems using incompatible methods and an unified solution is still lacking. (2) They do not have the flexibility to support variable key/value sizes and value calculations in the runtime given a fixed hardware design. (3) They require a quadratic hardware cost (O(V2)) for given parallelism V in most cases. To address above three limitations, we make the following efforts. (1) We present a one-for-all solution to support all interested MSOs based on a unified abstraction model “restricted zip machine (RZM)”. (2) We propose a set of composable and parallel primitives for RZM to provide the flexibility to support variable key/value sizes and value calculations. (3) We provide the hardware design to implement the proposed primitives using only O(Vlog(V)) resource. With the above techniques, a flexible and efficient solution for MSOs has been built. Our design can be used either as a drop-in replacement of the merge unit in prior accelerators to reduce the cost from O(V2) to O(Vlog(V)), or as an extension to the SIMD ISA of CPU and GPU. In our evaluation on CPU, when V=16 (512-bit SIMD, 32-bit element), we achieve significant speedup on a range of representative kernels including set operations (8.4×), database joins (7.3×), sparse vector/matrix/tensor addition/multiplication on real/complex numbers (6.5×), merge sort (8.0× over scalar, 3.4× over the state-of-the-art SIMD), and SpGEMM (4.4× over the best one in the baseline collection). @InProceedings{ASPLOS22p669, author = {Bangyan Wang and Lei Deng and Fei Sun and Guohao Dai and Liu Liu and Yu Wang and Yuan Xie}, title = {A One-for-All and <i>O</i>(<i>V</i> log(<i>V</i> ))-Cost Solution for Parallel Merge Style Operations on Sorted Key-Value Arrays}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {669--682}, doi = {10.1145/3503222.3507728}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Sun, Xian-He	ASPLOS '22: "NVAlloc: Rethinking Heap Metadata ..." NVAlloc: Rethinking Heap Metadata Management in Persistent Memory Allocators Zheng Dang, Shuibing He, Peiyi Hong, Zhenxin Li, Xuechen Zhang, Xian-He Sun, and Gang Chen (Zhejiang University, China; Washington State University, USA; Illinois Institute of Technology, USA) Persistent memory allocation is a fundamental building block for developing high-performance and in-memory applications. Existing persistent memory allocators suffer from suboptimal heap organizations that introduce repeated cache line flushes and small random accesses in persistent memory. Worse, many allocators use static slab segregation resulting in a dramatic increase in memory consumption when allocation request size is changed. In this paper, we design a novel allocator, named NVAlloc, to solve the above issues simultaneously. First, NVAlloc eliminates cache line reflushes by mapping contiguous data blocks in slabs to interleaved metadata entries stored in different cache lines. Second, it writes small metadata units to a persistent bookkeeping log in a sequential pattern to remove random heap metadata accesses in persistent memory. Third, instead of using static slab segregation, it supports slab morphing, which allows slabs to be transformed between size classes to significantly improve slab usage. NVAlloc is complementary to the existing consistency models. Results on 6 benchmarks demonstrate that NVAlloc improves the performance of state-of-the-art persistent memory allocators by up to 6.4x and 57x for small and large allocations, respectively. Using NVAlloc reduces memory usage by up to 57.8%. Besides, we integrate NVAlloc in a persistent FPTree. Compared to the state-of-the-art allocators, NVAlloc improves the performance of this application by up to 3.1x. @InProceedings{ASPLOS22p115, author = {Zheng Dang and Shuibing He and Peiyi Hong and Zhenxin Li and Xuechen Zhang and Xian-He Sun and Gang Chen}, title = {NVAlloc: Rethinking Heap Metadata Management in Persistent Memory Allocators}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {115--127}, doi = {10.1145/3503222.3507743}, year = {2022}, } Publisher's Version
	Swamy, Tushar	ASPLOS '22: "Taurus: A Data Plane Architecture ..." Taurus: A Data Plane Architecture for Per-Packet ML Tushar Swamy, Alexander Rucker, Muhammad Shahbaz, Ishan Gaur, and Kunle Olukotun (Stanford University, USA; Purdue University, USA) Emerging applications---cloud computing, the internet of things, and augmented/virtual reality---demand responsive, secure, and scalable datacenter networks. These networks currently implement simple, per-packet, data-plane heuristics (e.g., ECMP and sketches) under a slow, millisecond-latency control plane that runs data-driven performance and security policies. However, to meet applications' service-level objectives (SLOs) in a modern data center, networks must bridge the gap between line-rate, per-packet execution and complex decision making. In this work, we present the design and implementation of Taurus, a data plane for line-rate inference. Taurus adds custom hardware based on a flexible, parallel-patterns (MapReduce) abstraction to programmable network devices, such as switches and NICs; this new hardware uses pipelined SIMD parallelism to enable per-packet MapReduce operations (e.g., inference). Our evaluation of a Taurus switch ASIC---supporting several real-world models---shows that Taurus operates orders of magnitude faster than a server-based control plane while increasing area by 3.8% and latency for line-rate ML models by up to 221 ns. Furthermore, our Taurus FPGA prototype achieves full model accuracy and detects two orders of magnitude more events than a state-of-the-art control-plane anomaly-detection system. @InProceedings{ASPLOS22p1099, author = {Tushar Swamy and Alexander Rucker and Muhammad Shahbaz and Ishan Gaur and Kunle Olukotun}, title = {Taurus: A Data Plane Architecture for Per-Packet ML}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1099--1114}, doi = {10.1145/3503222.3507726}, year = {2022}, } Publisher's Version Published Artifact Info Artifacts Available
	Swift, Michael	ASPLOS '22: "Software-Defined Address Mapping: ..." Software-Defined Address Mapping: A Case on 3D Memory Jialiang Zhang, Michael Swift, and Jing (Jane) Li (University of Pennsylvania, USA; University of Wisconsin-Madison, USA) 3D-stacking memory such as High-Bandwidth Memory (HBM) and Hybrid Memory Cube (HMC) provides orders of magnitude more bandwidth and significantly increased channel-level parallelism (CLP) due to its new parallel memory architecture. However, it is challenging to fully exploit the abundant CLP for performance as the bandwidth utilization is highly dependent on address mapping in the memory controller. Unfortunately, CLP is very sensitive to a program’s data access pattern, which is not made available to OS/hardware by existing mechanisms. In this work, we address these challenges with software-defined address mapping (SDAM) that, for the first time, enables user program to obtain a direct control of the low-level memory hardware in a more intelligent and fine-grained manner. In particular, we develop new mechanisms that can effectively communicate a program’s data access properties to the OS and hardware and to use it to control data placement in hardware. To guarantee correctness and reduce overhead in storage and performance, we extend Linux kernel and C-language memory allocators to support multiple address mappings. For advanced system optimization, we develop machine learning methods that can automatically identify access patterns of major variables in a program and cluster these with similar access patterns to reduce the overhead for SDAM. We demonstrate the benefits of our design on real system prototype, comprising (1) a RISC-V processor, near memory accelerators and HBM modules using Xilinx FPGA platform, and (2) modified Linux and glibc. Our evaluation on standard CPU benchmarks and data-intensive benchmarks (for both CPU and accelerators) demonstrates 1.41×, 1.84× speedup on CPU and 2.58× on near memory accelerators in our system with SDAM compared to a baseline system that uses a fixed address mapping. @InProceedings{ASPLOS22p70, author = {Jialiang Zhang and Michael Swift and Jing (Jane) Li}, title = {Software-Defined Address Mapping: A Case on 3D Memory}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {70--83}, doi = {10.1145/3503222.3507774}, year = {2022}, } Publisher's Version
	Tang, Chunqiang	ASPLOS '22: "IOCost: Block IO Control for ..." IOCost: Block IO Control for Containers in Datacenters Tejun Heo, Dan Schatzberg, Andrew Newell, Song Liu, Saravanan Dhakshinamurthy, Iyswarya Narayanan, Josef Bacik, Chris Mason, Chunqiang Tang, and Dimitrios Skarlatos (Meta, USA; Carnegie Mellon University, USA) Resource isolation is a fundamental requirement in datacenter environments. However, our production experience in Meta’s large-scale datacenters shows that existing IO control mechanisms for block storage are inadequate in containerized environments. IO control needs to provide proportional resources to containers while taking into account the hardware heterogeneity of storage devices and the idiosyncrasies of the workloads deployed in datacenters. The speed of modern SSDs requires IO control to execute with low-overheads. Furthermore, IO control should strive for work conservation, take into account the interactions with the memory management subsystem, and avoid priority inversions that lead to isolation failures. To address these challenges, this paper presents IOCost, an IO control solution that is designed for containerized environments and provides scalable, work-conserving, and low-overhead IO control for heterogeneous storage devices and diverse workloads in datacenters. IOCost performs offline profiling to build a device model and uses it to estimate device occupancy of each IO request. To minimize runtime overhead, it separates IO control into a fast per-IO issue path and a slower periodic planning path. A novel work-conserving budget donation algorithm enables containers to dynamically share unused budget. We have deployed IOCost across the entirety of Meta’s datacenters comprised of millions of ma- chines, upstreamed IOCost to the Linux kernel, and open-sourced our device-profiling tools. IOCost has been running in production for two years, providing IO control for Meta’s fleet. We describe the design of IOCost and share our experience deploying it at scale. @InProceedings{ASPLOS22p595, author = {Tejun Heo and Dan Schatzberg and Andrew Newell and Song Liu and Saravanan Dhakshinamurthy and Iyswarya Narayanan and Josef Bacik and Chris Mason and Chunqiang Tang and Dimitrios Skarlatos}, title = {IOCost: Block IO Control for Containers in Datacenters}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {595--608}, doi = {10.1145/3503222.3507727}, year = {2022}, } Publisher's Version ASPLOS '22: "TMO: Transparent Memory Offloading ..." TMO: Transparent Memory Offloading in Datacenters Johannes Weiner, Niket Agarwal, Dan Schatzberg, Leon Yang, Hao Wang, Blaise Sanouillet, Bikash Sharma, Tejun Heo, Mayank Jain, Chunqiang Tang, and Dimitrios Skarlatos (Meta, USA; Carnegie Mellon University, USA) The unrelenting growth of the memory needs of emerging datacenter applications, along with ever increasing cost and volatility of DRAM prices, has led to DRAM being a major infrastructure expense. Alternative technologies, such as NVMe SSDs and upcoming NVM devices, offer higher capacity than DRAM at a fraction of the cost and power. One promising approach is to transparently offload colder memory to cheaper memory technologies via kernel or hypervisor techniques. The key challenge, however, is to develop a datacenter-scale solution that is robust in dealing with diverse workloads and large performance variance of different offload devices such as compressed memory, SSD, and NVM. This paper presents TMO, Meta’s transparent memory offloading solution for heterogeneous datacenter environments. TMO introduces a new Linux kernel mechanism that directly measures in realtime the lost work due to resource shortage across CPU, memory, and I/O. Guided by this information and without any prior application knowledge, TMO automatically adjusts how much memory to offload to heterogeneous devices (e.g., compressed memory or SSD) according to the device’s performance characteristics and the application’s sensitivity to memory-access slowdown. TMO holistically identifies offloading opportunities from not only the application containers but also the sidecar containers that provide infrastructure-level functions. To maximize memory savings, TMO targets both anonymous memory and file cache, and balances the swap-in rate of anonymous memory and the reload rate of file pages that were recently evicted from the file cache. TMO has been running in production for more than a year, and has saved between 20-32% of the total memory across millions of servers in our large datacenter fleet. We have successfully upstreamed TMO into the Linux kernel. @InProceedings{ASPLOS22p609, author = {Johannes Weiner and Niket Agarwal and Dan Schatzberg and Leon Yang and Hao Wang and Blaise Sanouillet and Bikash Sharma and Tejun Heo and Mayank Jain and Chunqiang Tang and Dimitrios Skarlatos}, title = {TMO: Transparent Memory Offloading in Datacenters}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {609--621}, doi = {10.1145/3503222.3507731}, year = {2022}, } Publisher's Version
	Tannu, Swamit	ASPLOS '22: "HAMMER: Boosting Fidelity ..." HAMMER: Boosting Fidelity of Noisy Quantum Circuits by Exploiting Hamming Behavior of Erroneous Outcomes Swamit Tannu, Poulami Das, Ramin Ayanzadeh, and Moinuddin Qureshi (University of Wisconsin-Madison, USA; Georgia Institute of Technology, USA) Quantum computers with hundreds of qubits will be available soon. Unfortunately, high device error-rates pose a significant challenge in using these near-term quantum systems to power real-world applications. Executing a program on existing quantum systems generates both correct and incorrect outcomes, but often, the output distribution is too noisy to distinguish between them. In this paper, we show that erroneous outcomes are not arbitrary but exhibit a well-defined structure when represented in the Hamming space. Our experiments on IBM and Google quantum computers show that the most frequent erroneous outcomes are more likely to be close in the Hamming space to the correct outcome. We exploit this behavior to improve the ability to infer the correct outcome. We propose Hamming Reconstruction (HAMMER), a post-processing technique that leverages the observation of Hamming behavior to reconstruct the noisy output distribution, such that the resulting distribution has higher fidelity. We evaluate HAMMER using experimental data from Google and IBM quantum computers with more than 500 unique quantum circuits and obtain an average improvement of 1.37x in the quality of solution. On Google’s publicly available QAOA datasets, we show that HAMMER sharpens the gradients on the cost function landscape. @InProceedings{ASPLOS22p529, author = {Swamit Tannu and Poulami Das and Ramin Ayanzadeh and Moinuddin Qureshi}, title = {HAMMER: Boosting Fidelity of Noisy Quantum Circuits by Exploiting Hamming Behavior of Erroneous Outcomes}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {529--540}, doi = {10.1145/3503222.3507703}, year = {2022}, } Publisher's Version
	Teodorescu, Stefan Lucian	ASPLOS '22: "FlexOS: Towards Flexible OS ..." FlexOS: Towards Flexible OS Isolation Hugo Lefeuvre, Vlad-Andrei Bădoiu, Alexander Jung, Stefan Lucian Teodorescu, Sebastian Rauch, Felipe Huici, Costin Raiciu, and Pierre Olivier (University of Manchester, UK; University Politehnica of Bucharest, Romania; Lancaster University, UK; unikraft.io, Germany; KIT, Germany; NEC Laboratories, Germany; Correct Networks, Romania) At design time, modern operating systems are locked in a specific safety and isolation strategy that mixes one or more hardware/software protection mechanisms (e.g. user/kernel separation); revisiting these choices after deployment requires a major refactoring effort. This rigid approach shows its limits given the wide variety of modern applications' safety/performance requirements, when new hardware isolation mechanisms are rolled out, or when existing ones break. We present FlexOS, a novel OS allowing users to easily specialize the safety and isolation strategy of an OS at compilation/deployment time instead of design time. This modular LibOS is composed of fine-grained components that can be isolated via a range of hardware protection mechanisms with various data sharing strategies and additional software hardening. The OS ships with an exploration technique helping the user navigate the vast safety/performance design space it unlocks. We implement a prototype of the system and demonstrate, for several applications (Redis/Nginx/SQLite), FlexOS' vast configuration space as well as the efficiency of the exploration technique: we evaluate 80 FlexOS configurations for Redis and show how that space can be probabilistically subset to the 5 safest ones under a given performance budget. We also show that, under equivalent configurations, FlexOS performs similarly or better than existing solutions which use fixed safety configurations. @InProceedings{ASPLOS22p467, author = {Hugo Lefeuvre and Vlad-Andrei Bădoiu and Alexander Jung and Stefan Lucian Teodorescu and Sebastian Rauch and Felipe Huici and Costin Raiciu and Pierre Olivier}, title = {FlexOS: Towards Flexible OS Isolation}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {467--482}, doi = {10.1145/3503222.3507759}, year = {2022}, } Publisher's Version Published Artifact Info Artifacts Available Artifacts Functional Results Reproduced
	Theodoridis, Theodoros	ASPLOS '22: "Finding Missed Optimizations ..." Finding Missed Optimizations through the Lens of Dead Code Elimination Theodoros Theodoridis, Manuel Rigger, and Zhendong Su (ETH Zurich, Switzerland) Compilers are foundational software development tools and incorporate increasingly sophisticated optimizations. Due to their complexity, it is difficult to systematically identify opportunities for improving them. Indeed, the automatic discovery of missed optimizations has been an important and significant challenge. The few existing approaches either cannot accurately pinpoint missed optimizations or target only specific analyses. This paper tackles this challenge by introducing a novel, effective approach that --- in a simple and general manner --- automatically identifies a wide range of missed optimizations. Our core insight is to leverage dead code elimination (DCE) to both analyze how well compilers optimize code and identify missed optimizations: (1) insert "optimization markers" in the basic blocks of a given program, (2) compute the program's live/dead basic blocks using the "optimization markers", and (3) identify missed optimizations from how well compilers eliminate dead blocks. We essentially exploit that, since DCE heavily depends on the rest of the optimization pipeline, through the lens of DCE, one can systematically quantify how well compilers optimize code. We conduct an extensive analysis of GCC and LLVM using our approach, which (1) provides quantitative and qualitative insights regarding their optimization capabilities, and (2) uncovers a diverse set of missed optimizations. Our results also lead to 84 bug reports for GCC and LLVM, of which 62 have already been confirmed or fixed, demonstrating our work's strong practical utility. We expect that the simplicity and generality of our approach will make it widely applicable for understanding compiler performance and finding missed optimizations. This work opens and initiates this promising direction. @InProceedings{ASPLOS22p697, author = {Theodoros Theodoridis and Manuel Rigger and Zhendong Su}, title = {Finding Missed Optimizations through the Lens of Dead Code Elimination}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {697--709}, doi = {10.1145/3503222.3507764}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced ASPLOS '22: "Understanding and Exploiting ..." Understanding and Exploiting Optimal Function Inlining Theodoros Theodoridis, Tobias Grosser, and Zhendong Su (ETH Zurich, Switzerland; University of Edinburgh, UK) Inlining is a core transformation in optimizing compilers. It replaces a function call (call site) with the body of the called function (callee). It helps reduce function call overhead and binary size, and more importantly, enables other optimizations. The problem of inlining has been extensively studied, but it is far from being solved; predicting which inlining decisions are beneficial is nontrivial due to interactions with the rest of the compiler pipeline. Previous work has mainly focused on designing heuristics for better inlining decisions and has not investigated optimal inlining, i.e., exhaustively finding the optimal inlining decisions. Optimal inlining is necessary for identifying and exploiting missed opportunities and evaluating the state of the art. This paper fills this gap through an extensive empirical analysis of optimal inlining using the SPEC2017 benchmark suite. Our novel formulation drastically reduces the inlining search space size (from 2349 down to 225) and allows us to exhaustively evaluate all inlining choices on 1,135 SPEC2017 files. We show a significant gap between the state-of-the-art strategy in LLVM and optimal inlining when optimizing for binary size, an important, deterministic metric independent of workload (in contrast to performance, another important metric). Inspired by our analysis, we introduce a simple, effective autotuning strategy for inlining that outperforms the state of the art by 7% on average (and up to 28%) on SPEC2017, 15% on the source code of LLVM itself, and 10% on the source code of SQLite. This work highlights the importance of exploring optimal inlining by providing new, actionable insight and an effective autotuning strategy that is of practical utility. @InProceedings{ASPLOS22p977, author = {Theodoros Theodoridis and Tobias Grosser and Zhendong Su}, title = {Understanding and Exploiting Optimal Function Inlining}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {977--989}, doi = {10.1145/3503222.3507744}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Tian, Boyu	ASPLOS '22: "FINGERS: Exploiting Fine-Grained ..." FINGERS: Exploiting Fine-Grained Parallelism in Graph Mining Accelerators Qihang Chen, Boyu Tian, and Mingyu Gao (Tsinghua University, China) Graph mining is an emerging application of high importance and also with high complexity, thus requiring efficient hardware acceleration. Current accelerator designs only utilize coarse-grained parallelism, leaving large room for further optimizations. Our key insight is to fully exploit fine-grained parallelism to overcome the existing issues of hardware underutilization, inefficient resource provision, and limited single-thread performance under imbalanced loads. Targeting pattern-aware graph mining algorithms, we first comprehensively identify and analyze the abundant fine-grained parallelism at the branch, set, and segment levels during search tree exploration and set operations. We then propose a novel graph mining accelerator, FINGERS, which effectively exploits these multiple levels of fine-grained parallelism to achieve significant performance improvements. FINGERS mainly enhances the design of each single processing element with parallel compute units for set operations, and efficient techniques for task scheduling, load balancing, and data aggregation. FINGERS outperforms the state-of-the-art design by 2.8× on average and up to 8.9× with the same chip area. We also demonstrate that different patterns and different graphs exhibit drastically different parallelism opportunities, justifying the necessity of exploiting all levels of fine-grained parallelism in FINGERS. @InProceedings{ASPLOS22p43, author = {Qihang Chen and Boyu Tian and Mingyu Gao}, title = {FINGERS: Exploiting Fine-Grained Parallelism in Graph Mining Accelerators}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {43--55}, doi = {10.1145/3503222.3507730}, year = {2022}, } Publisher's Version
	Tiwari, Devesh	ASPLOS '22: "QUEST: Systematically Approximating ..." QUEST: Systematically Approximating Quantum Circuits for Higher Output Fidelity Tirthak Patel, Ed Younis, Costin Iancu, Wibe de Jong, and Devesh Tiwari (Northeastern University, USA; Lawrence Berkeley National Laboratory, USA) We present QUEST, a procedure to systematically generate approximations for quantum circuits to reduce their CNOT gate count. Our approach employs circuit partitioning for scalability with procedures to 1) reduce circuit length using approximate synthesis, 2) improve fidelity by running circuits that represent key samples in the approximation space, and 3) reason about approximation upper bound. Our evaluation results indicate that our approach of "dissimilar" approximations provides close fidelity to the original circuit. Overall, the results indicate that QUEST can reduce CNOT gate count by 30-80% on ideal systems and decrease the impact of noise on existing and near-future quantum systems. @InProceedings{ASPLOS22p514, author = {Tirthak Patel and Ed Younis and Costin Iancu and Wibe de Jong and Devesh Tiwari}, title = {QUEST: Systematically Approximating Quantum Circuits for Higher Output Fidelity}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {514--528}, doi = {10.1145/3503222.3507739}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional ASPLOS '22: "IceBreaker: Warming Serverless ..." IceBreaker: Warming Serverless Functions Better with Heterogeneity Rohan Basu Roy, Tirthak Patel, and Devesh Tiwari (Northeastern University, USA) Serverless computing, an emerging computing model, relies on "warming up" functions prior to its anticipated execution for faster and cost-effective service to users. Unfortunately, warming up functions can be inaccurate and incur prohibitively expensive cost during the warmup period (i.e., keep-alive cost). In this paper, we introduce IceBreaker, a novel technique that reduces the service time and the "keep-alive" cost by composing a system with heterogeneous nodes (costly and cheaper). IceBreaker does so by dynamically determining the cost-effective node type to warm up a function based on the function's time-varying probability of the next invocation. By employing heterogeneity, IceBreaker allows for more number of nodes under the same cost budget and hence, keeps more number of functions warm and reduces the wait time during high load. Our real-system evaluation confirms that IceBreaker reduces the overall keep-alive cost by 45% and execution time by 27% using representative serverless applications and industry-grade workload trace. IceBreaker is the first technique to employ and leverage the idea of mixing expensive and cheaper nodes to improve both service time and keep-alive cost for serverless functions -- opening up a new research avenue of serverless computing on heterogeneous servers for researchers and practitioners. @InProceedings{ASPLOS22p753, author = {Rohan Basu Roy and Tirthak Patel and Devesh Tiwari}, title = {IceBreaker: Warming Serverless Functions Better with Heterogeneity}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {753--767}, doi = {10.1145/3503222.3507750}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Torrellas, Josep	ASPLOS '22: "Pinned Loads: Taming Speculative ..." Pinned Loads: Taming Speculative Loads in Secure Processors Zirui Neil Zhao, Houxiang Ji, Adam Morrison, Darko Marinov, and Josep Torrellas (University of Illinois at Urbana-Champaign, USA; Tel Aviv University, Israel) In security frameworks for speculative execution, an instruction is said to reach its Visibility Point (VP) when it is no longer vulnerable to pipeline squashes. Before a potentially leaky instruction reaches its VP, it has to stall—unless a defense scheme such as invisible speculation provides protection. Unfortunately, either stalling or protecting the execution of pre-VP instructions typically has a performance cost. One way to attain low-overhead safe execution is to develop techniques that speed-up the advance of the VP from older to younger instructions. In this paper, we propose one such technique. We find that the progress of the VP for loads is mostly impeded by waiting until no memory consistency violations (MCVs) are possible. Hence, our technique, called , tries to make loads invulnerable to MCVs as early as possible—a process we call pinning the loads in the pipeline. The result is faster VP progress and a reduction in the execution overhead of defense schemes. In this paper, we describe the hardware needed by , and two possible designs with different tradeoffs between hardware requirements and performance. Our evaluation shows that is very effective: extending three popular defense schemes against speculative execution attacks with reduces their average execution overhead on SPEC17 and on SPLASH2/PARSEC applications by about 50%. For example, on SPEC17, the execution overhead of the three defense schemes decreases from to , from to , and from to . @InProceedings{ASPLOS22p314, author = {Zirui Neil Zhao and Houxiang Ji and Adam Morrison and Darko Marinov and Josep Torrellas}, title = {Pinned Loads: Taming Speculative Loads in Secure Processors}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {314--328}, doi = {10.1145/3503222.3507724}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced ASPLOS '22: "Parallel Virtualized Memory ..." Parallel Virtualized Memory Translation with Nested Elastic Cuckoo Page Tables Jovan Stojkovic, Dimitrios Skarlatos, Apostolos Kokolis, Tianyin Xu, and Josep Torrellas (University of Illinois at Urbana-Champaign, USA; Carnegie Mellon University, USA) A major reason why nested or virtualized address translations are slow is because current systems organize page tables in a multi-level tree that is accessed in a sequential manner. A nested translation may potentially require up to twenty-four sequential memory accesses. To address this problem, this paper presents the first page table design that supports parallel nested address translation. The design is based on using hashed page tables (HPTs) for both guest and host. However, directly extending a native HPT design to a nested environment leads to minor gains. Instead, our design solves a new set of challenges that appear in nested environments. Our scheme eliminates all but three of the potentially twenty-four sequential steps of a nested translation—while judiciously limiting the number of parallel memory accesses issued to avoid over-consuming cache bandwidth. As a result, compared to conventional nested radix tables, our design speeds-up the execution of a set of applications by an average of 1.19x (for 4KB pages) and 1.24x (when huge pages are used). In addition, we also show a migration path from current nested radix page tables to our design. @InProceedings{ASPLOS22p84, author = {Jovan Stojkovic and Dimitrios Skarlatos and Apostolos Kokolis and Tianyin Xu and Josep Torrellas}, title = {Parallel Virtualized Memory Translation with Nested Elastic Cuckoo Page Tables}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {84--97}, doi = {10.1145/3503222.3507720}, year = {2022}, } Publisher's Version
	Trippel, Caroline	ASPLOS '22: "RecShard: Statistical Feature-Based ..." RecShard: Statistical Feature-Based Memory Optimization for Industry-Scale Neural Recommendation Geet Sethi, Bilge Acun, Niket Agarwal, Christos Kozyrakis, Caroline Trippel, and Carole-Jean Wu (Stanford University, USA; Meta, USA) We propose RecShard, a fine-grained embedding table (EMB) partitioning and placement technique for deep learning recommendation models (DLRMs). RecShard is designed based on two key observations. First, not all EMBs are equal, nor all rows within an EMB are equal in terms of access patterns. EMBs exhibit distinct memory characteristics, providing performance optimization opportunities for intelligent EMB partitioning and placement across a tiered memory hierarchy. Second, in modern DLRMs, EMBs function as hash tables. As a result, EMBs display interesting phenomena, such as the birthday paradox, leaving EMBs severely under-utilized. RecShard determines an optimal EMB sharding strategy for a set of EMBs based on training data distributions and model characteristics, along with the bandwidth characteristics of the underlying tiered memory hierarchy. In doing so, RecShard achieves over 6 times higher EMB training throughput on average for capacity constrained DLRMs. The throughput increase comes from improved EMB load balance by over 12 times and from the reduced access to the slower memory by over 87 times. @InProceedings{ASPLOS22p344, author = {Geet Sethi and Bilge Acun and Niket Agarwal and Christos Kozyrakis and Caroline Trippel and Carole-Jean Wu}, title = {RecShard: Statistical Feature-Based Memory Optimization for Industry-Scale Neural Recommendation}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {344--358}, doi = {10.1145/3503222.3507777}, year = {2022}, } Publisher's Version
	Tsafrir, Dan	ASPLOS '22: "The Benefits of General-Purpose ..." The Benefits of General-Purpose On-NIC Memory Boris Pismenny, Liran Liss, Adam Morrison, and Dan Tsafrir (Technion, Israel; NVIDIA, Israel; Tel Aviv University, Israel; VMware Research, Israel) We propose to use the small, newly available on-NIC memory ("nicmem") to keep pace with the rapidly increasing performance of NICs. We motivate our proposal by accelerating two types of workload classes: NFV and key-value stores. As NFV workloads frequently operate on headers---rather than data---of incoming packets, we introduce a new packet-processing architecture that splits between the two, keeping the data on nicmem when possible and thus reducing PCIe traffic, memory bandwidth, and CPU processing time. Our approach consequently shortens NFV latency by up to 23% and increases its throughput by up to 19%. Similarly, because key-value stores commonly exhibit skewed distributions, we introduce a new network stack mechanism that lets applications keep frequently accessed items on nicmem. Our design shortens memcached latency by up to 43% and increases its throughput by up to 80%. @InProceedings{ASPLOS22p1130, author = {Boris Pismenny and Liran Liss and Adam Morrison and Dan Tsafrir}, title = {The Benefits of General-Purpose On-NIC Memory}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1130--1147}, doi = {10.1145/3503222.3507711}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Tu, Fengbin	ASPLOS '22: "DOTA: Detect and Omit Weak ..." DOTA: Detect and Omit Weak Attentions for Scalable Transformer Acceleration Zheng Qu, Liu Liu, Fengbin Tu, Zhaodong Chen, Yufei Ding, and Yuan Xie (University of California at Santa Barbara, USA) Transformer Neural Networks have demonstrated leading performance in many applications spanning over language understanding, image processing, and generative modeling. Despite the impressive performance, long-sequence Transformer processing is expensive due to quadratic computation complexity and memory consumption of self-attention. In this paper, we present DOTA, an algorithm-architecture co-design that effectively addresses the challenges of scalable Transformer inference. Based on the insight that not all connections in an attention graph are equally important, we propose to jointly optimize a lightweight Detector with the Transformer model to accurately detect and omit weak connections during runtime. Furthermore, we design a specialized system architecture for end-to-end Transformer acceleration using the proposed attention detection mechanism. Experiments on a wide range of benchmarks demonstrate the superior performance of DOTA over other solutions. In summary, DOTA achieves 152.6x and 4.5x performance speedup and orders of magnitude energy-efficiency improvements over GPU and customized hardware, respectively. @InProceedings{ASPLOS22p14, author = {Zheng Qu and Liu Liu and Fengbin Tu and Zhaodong Chen and Yufei Ding and Yuan Xie}, title = {DOTA: Detect and Omit Weak Attentions for Scalable Transformer Acceleration}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {14--26}, doi = {10.1145/3503222.3507738}, year = {2022}, } Publisher's Version
	Tunç, Hünkar Can	ASPLOS '22: "A Tree Clock Data Structure ..." A Tree Clock Data Structure for Causal Orderings in Concurrent Executions Umang Mathur, Andreas Pavlogiannis, Hünkar Can Tunç, and Mahesh Viswanathan (National University of Singapore, Singapore; Aarhus University, Denmark; University of Illinois at Urbana-Champaign, USA) Dynamic techniques are a scalable and effective way to analyze concurrent programs. Instead of analyzing all behaviors of a program, these techniques detect errors by focusing on a single program execution. Often a crucial step in these techniques is to define a causal ordering between events in the execution, which is then computed using vector clocks, a simple data structure that stores logical times of threads. The two basic operations of vector clocks, namely join and copy, require Θ(k) time, where k is the number of threads. Thus they are a computational bottleneck when k is large. In this work, we introduce tree clocks, a new data structure that replaces vector clocks for computing causal orderings in program executions. Joining and copying tree clocks takes time that is roughly proportional to the number of entries being modified, and hence the two operations do not suffer the a-priori Θ(k) cost per application. We show that when used to compute the classic happens-before (HB) partial order, tree clocks are optimal, in the sense that no other data structure can lead to smaller asymptotic running time. Moreover, we demonstrate that tree clocks can be used to compute other partial orders, such as schedulable-happens-before (SHB) and the standard Mazurkiewicz (MAZ) partial order, and thus are a versatile data structure. Our experiments show that just by replacing vector clocks with tree clocks, the computation becomes from 2.02 × faster (MAZ) to 2.66 × (SHB) and 2.97 × (HB) on average per benchmark. These results illustrate that tree clocks have the potential to become a standard data structure with wide applications in concurrent analyses. @InProceedings{ASPLOS22p710, author = {Umang Mathur and Andreas Pavlogiannis and Hünkar Can Tunç and Mahesh Viswanathan}, title = {A Tree Clock Data Structure for Causal Orderings in Concurrent Executions}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {710--725}, doi = {10.1145/3503222.3507734}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Turowski, Adam	ASPLOS '22: "Enzian: An Open, General, ..." Enzian: An Open, General, CPU/FPGA Platform for Systems Software Research David Cock, Abishek Ramdas, Daniel Schwyn, Michael Giardino, Adam Turowski, Zhenhao He, Nora Hossle, Dario Korolija, Melissa Licciardello, Kristina Martsenko, Reto Achermann, Gustavo Alonso, and Timothy Roscoe (ETH Zurich, Switzerland; University of British Columbia, Canada) Hybrid computing platforms, comprising CPU cores and FPGA logic, are increasingly used for accelerating data-intensive workloads in cloud deployments, and are a growing topic of interest in systems research. However, from a research perspective, existing hardware platforms are limited: they are often optimized for concrete, narrow use-cases and, therefore lack the flexibility needed to explore other applications and configurations. We show that a research group can design and build a more general, open, and affordable hardware platform for hybrid systems research. The platform, Enzian, is capable of duplicating the functionality of existing CPU/FPGA systems with comparable performance but in an open, flexible system. It couples a large FPGA with a server-class CPU in an asymmetric cache-coherent NUMA system. Enzian also enables research not possible with existing hybrid platforms, through explicit access to coherence messages, extensive thermal and power instrumentation, and an open, programmable baseboard management processor. Enzian is already being used in multiple projects, is open source (both hardware and software), and available for remote use. We present the design principles of Enzian, the challenges in building it, and evaluate it with a range of existing research use-cases alongside other, more specialized platforms, as well as demonstrating research not possible on existing platforms. @InProceedings{ASPLOS22p434, author = {David Cock and Abishek Ramdas and Daniel Schwyn and Michael Giardino and Adam Turowski and Zhenhao He and Nora Hossle and Dario Korolija and Melissa Licciardello and Kristina Martsenko and Reto Achermann and Gustavo Alonso and Timothy Roscoe}, title = {Enzian: An Open, General, CPU/FPGA Platform for Systems Software Research}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {434--451}, doi = {10.1145/3503222.3507742}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Unsal, Osman Sabri	ASPLOS '22: "BiSon-e: A Lightweight and ..." BiSon-e: A Lightweight and High-Performance Accelerator for Narrow Integer Linear Algebra Computing on the Edge Enrico Reggiani, Cristóbal Ramírez Lazo, Roger Figueras Bagué, Adrián Cristal, Mauro Olivieri, and Osman Sabri Unsal (Polytechnic University of Catalonia, Spain; Barcelona Supercomputing Center, Spain; Sapienza University of Rome, Italy) Linear algebra computational kernels based on byte and sub-byte integer data formats are at the base of many classes of applications, ranging from Deep Learning to Pattern Matching. Porting the computation of these applications from cloud to edge and mobile devices would enable significant improvements in terms of security, safety, and energy efficiency. However, despite their low memory and energy demands, their intrinsically high computational intensity makes the execution of these workloads challenging on highly resource-constrained devices. In this paper, we present BiSon-e, a novel RISC-V based architecture that accelerates linear algebra kernels based on narrow integer computations on edge processors by performing Single Instruction Multiple Data (SIMD) operations on off-the-shelf scalar Functional Units (FUs). Our novel architecture is built upon the binary segmentation technique, which allows to significantly reduce the memory footprint and the arithmetic intensity of linear algebra kernels requiring narrow data sizes. We integrate BiSon-e into a complete System-on-Chip (SoC) based on RISC-V, synthesized and Place&Routed in 65nm and 22nm technologies, introducing a negligible 0.07% area overhead with respect to the baseline architecture. Our experimental evaluation shows that, when computing the Convolution and Fully-Connected layers of the AlexNet and VGG-16 Convolutional Neural Networks (CNNs) with 8-, 4-, and 2-bit, our solution gains up to 5.6×, 13.9× and 24× in execution time compared to the scalar implementation of a single RISC-V core, and improves the energy efficiency of string matching tasks by 5× when compared to a RISC-V-based Vector Processing Unit (VPU). @InProceedings{ASPLOS22p56, author = {Enrico Reggiani and Cristóbal Ramírez Lazo and Roger Figueras Bagué and Adrián Cristal and Mauro Olivieri and Osman Sabri Unsal}, title = {BiSon-e: A Lightweight and High-Performance Accelerator for Narrow Integer Linear Algebra Computing on the Edge}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {56--69}, doi = {10.1145/3503222.3507746}, year = {2022}, } Publisher's Version
	Van der Hagen, McKenzie	ASPLOS '22: "Client-Optimized Algorithms ..." Client-Optimized Algorithms and Acceleration for Encrypted Compute Offloading McKenzie van der Hagen and Brandon Lucia (Carnegie Mellon University, USA) Homomorphic Encryption (HE) enables secure cloud offload processing on encrypted data. HE schemes are limited in the complexity and type of operations they can perform, motivating client-aided implementations that distribute computation between client (unencrypted) and server (encrypted). Prior client-aided systems optimize server performance, ignoring client costs: client-aided models put encryption and decryption on the critical path and require communicating large ciphertexts. We introduce Client-aided HE for Opaque Compute Offloading (CHOCO), a client-optimized system for encrypted offload processing. CHOCO reduces ciphertext size, reducing communication and computing costs through HE parameter minimization and through “rotational redundancy”, a new HE algorithm optimization. We present Client-aided HE for Opaque Compute Offloading Through Accelerated Cryptographic Operations (CHOCO-TACO), an accelerator for HE encryption and decryption, making client-aided HE feasible for even resource-constrained clients. CHOCO supports two popular HE schemes (BFV and CKKS) and several applications, including DNNs, PageRank, KNN, and K-Means. CHOCO reduces communication by up to 2948× over prior work. With CHOCO-TACO client enc-/decryption is up to 1094× faster and uses up to 648× less energy. @InProceedings{ASPLOS22p683, author = {McKenzie van der Hagen and Brandon Lucia}, title = {Client-Optimized Algorithms and Acceleration for Encrypted Compute Offloading}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {683--696}, doi = {10.1145/3503222.3507737}, year = {2022}, } Publisher's Version
	Vijaykumar, Nandita	ASPLOS '22: "GenStore: A High-Performance ..." GenStore: A High-Performance In-Storage Processing System for Genome Sequence Analysis Nika Mansouri Ghiasi, Jisung Park, Harun Mustafa, Jeremie Kim, Ataberk Olgun, Arvid Gollwitzer, Damla Senol Cali, Can Firtina, Haiyu Mao, Nour Almadhoun Alserr, Rachata Ausavarungnirun, Nandita Vijaykumar, Mohammed Alser, and Onur Mutlu (ETH Zurich, Switzerland; Bionano Genomics, USA; King Mongkut's University of Technology North Bangkok, Thailand; University of Toronto, Canada) Read mapping is a fundamental step in many genomics applications. It is used to identify potential matches and differences between fragments (called reads) of a sequenced genome and an already known genome (called a reference genome). Read mapping is costly because it needs to perform approximate string matching (ASM) on large amounts of data. To address the computational challenges in genome analysis, many prior works propose various approaches such as accurate filters that select the reads within a dataset of genomic reads (called a read set) that must undergo expensive computation, efficient heuristics, and hardware acceleration. While effective at reducing the amount of expensive computation, all such approaches still require the costly movement of a large amount of data from storage to the rest of the system, which can significantly lower the end-to-end performance of read mapping in conventional and emerging genomics systems. We propose GenStore, the first in-storage processing system designed for genome sequence analysis that greatly reduces both data movement and computational overheads of genome sequence analysis by exploiting low-cost and accurate in-storage filters. GenStore leverages hardware/software co-design to address the challenges of in-storage processing, supporting reads with 1) different properties such as read lengths and error rates, which highly depend on the sequencing technology, and 2) different degrees of genetic variation compared to the reference genome, which highly depends on the genomes that are being compared. Through rigorous analysis of read mapping processes of reads with different properties and degrees of genetic variation, we meticulously design low-cost hardware accelerators and data/computation flows inside a NAND flash-based solid-state drive (SSD). Our evaluation using a wide range of real genomic datasets shows that GenStore, when implemented in three modern NAND flash-based SSDs, significantly improves the read mapping performance of state-of-the-art software (hardware) baselines by 2.07-6.05× (1.52-3.32×) for read sets with high similarity to the reference genome and 1.45-33.63× (2.70-19.2×) for read sets with low similarity to the reference genome. @InProceedings{ASPLOS22p635, author = {Nika Mansouri Ghiasi and Jisung Park and Harun Mustafa and Jeremie Kim and Ataberk Olgun and Arvid Gollwitzer and Damla Senol Cali and Can Firtina and Haiyu Mao and Nour Almadhoun Alserr and Rachata Ausavarungnirun and Nandita Vijaykumar and Mohammed Alser and Onur Mutlu}, title = {GenStore: A High-Performance In-Storage Processing System for Genome Sequence Analysis}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {635--654}, doi = {10.1145/3503222.3507702}, year = {2022}, } Publisher's Version
	Viswanathan, Mahesh	ASPLOS '22: "A Tree Clock Data Structure ..." A Tree Clock Data Structure for Causal Orderings in Concurrent Executions Umang Mathur, Andreas Pavlogiannis, Hünkar Can Tunç, and Mahesh Viswanathan (National University of Singapore, Singapore; Aarhus University, Denmark; University of Illinois at Urbana-Champaign, USA) Dynamic techniques are a scalable and effective way to analyze concurrent programs. Instead of analyzing all behaviors of a program, these techniques detect errors by focusing on a single program execution. Often a crucial step in these techniques is to define a causal ordering between events in the execution, which is then computed using vector clocks, a simple data structure that stores logical times of threads. The two basic operations of vector clocks, namely join and copy, require Θ(k) time, where k is the number of threads. Thus they are a computational bottleneck when k is large. In this work, we introduce tree clocks, a new data structure that replaces vector clocks for computing causal orderings in program executions. Joining and copying tree clocks takes time that is roughly proportional to the number of entries being modified, and hence the two operations do not suffer the a-priori Θ(k) cost per application. We show that when used to compute the classic happens-before (HB) partial order, tree clocks are optimal, in the sense that no other data structure can lead to smaller asymptotic running time. Moreover, we demonstrate that tree clocks can be used to compute other partial orders, such as schedulable-happens-before (SHB) and the standard Mazurkiewicz (MAZ) partial order, and thus are a versatile data structure. Our experiments show that just by replacing vector clocks with tree clocks, the computation becomes from 2.02 × faster (MAZ) to 2.66 × (SHB) and 2.97 × (HB) on average per benchmark. These results illustrate that tree clocks have the potential to become a standard data structure with wide applications in concurrent analyses. @InProceedings{ASPLOS22p710, author = {Umang Mathur and Andreas Pavlogiannis and Hünkar Can Tunç and Mahesh Viswanathan}, title = {A Tree Clock Data Structure for Causal Orderings in Concurrent Executions}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {710--725}, doi = {10.1145/3503222.3507734}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Vougioukas, Ilias	ASPLOS '22: "Every Walk’s a Hit: Making ..." Every Walk’s a Hit: Making Page Walks Single-Access Cache Hits Chang Hyun Park, Ilias Vougioukas, Andreas Sandberg, and David Black-Schaffer (Uppsala University, Sweden; Arm Research, UK) As memory capacity has outstripped TLB coverage, large data applications suffer from frequent page table walks. We investigate two complementary techniques for addressing this cost: reducing the number of accesses required and reducing the latency of each access. The first approach is accomplished by opportunistically "flattening" the page table: merging two levels of traditional 4 KB page table nodes into a single 2 MB node, thereby reducing the table's depth and the number of indirections required to traverse it. The second is accomplished by biasing the cache replacement algorithm to keep page table entries during periods of high TLB miss rates, as these periods also see high data miss rates and are therefore more likely to benefit from having the smaller page table in the cache than to suffer from increased data cache misses. We evaluate these approaches for both native and virtualized systems and across a range of realistic memory fragmentation scenarios, describe the limited changes needed in our kernel implementation and hardware design, identify and address challenges related to self-referencing page tables and kernel memory allocation, and compare results across server and mobile systems using both academic and industrial simulators for robustness. We find that flattening does reduce the number of accesses required on a page walk (to 1.0), but its performance impact (+2.3%) is small due to Page Walker Caches (already 1.5 accesses). Prioritizing caching has a larger effect (+6.8%), and the combination improves performance by +9.2%. Flattening is more effective on virtualized systems (4.4 to 2.8 accesses, +7.1% performance), due to 2D page walks. By combining the two techniques we demonstrate a state-of-the-art +14.0% performance gain and -8.7% dynamic cache energy and -4.7% dynamic DRAM energy for virtualized execution with very simple hardware and software changes. @InProceedings{ASPLOS22p128, author = {Chang Hyun Park and Ilias Vougioukas and Andreas Sandberg and David Black-Schaffer}, title = {Every Walk’s a Hit: Making Page Walks Single-Access Cache Hits}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {128--141}, doi = {10.1145/3503222.3507718}, year = {2022}, } Publisher's Version
	Wang, Bangyan	ASPLOS '22: "A One-for-All and O(V ..." A One-for-All and O(V log(V ))-Cost Solution for Parallel Merge Style Operations on Sorted Key-Value Arrays Bangyan Wang, Lei Deng, Fei Sun, Guohao Dai, Liu Liu, Yu Wang, and Yuan Xie (University of California at Santa Barbara, USA; Tsinghua University, China; Alibaba DAMO Academy, China) The processing of sorted key-value arrays using a “merge style operation (MSO)” is a very basic and important problem in domains like scientific computing, deep learning, database, graph analysis, sorting, set-operation etc. MSOs dominate the execution time in some important applications like SpGEMM and graph mining. For example, sparse vector addition as an MSO takes up to 98% execution time in SpGEMM in our experiment. For this reason, accelerating MSOs on CPU, GPU, and accelerators using parallel execution has been extensively studied but the solutions in prior work have three major limitations. (1) They treat different MSOs as isolated problems using incompatible methods and an unified solution is still lacking. (2) They do not have the flexibility to support variable key/value sizes and value calculations in the runtime given a fixed hardware design. (3) They require a quadratic hardware cost (O(V2)) for given parallelism V in most cases. To address above three limitations, we make the following efforts. (1) We present a one-for-all solution to support all interested MSOs based on a unified abstraction model “restricted zip machine (RZM)”. (2) We propose a set of composable and parallel primitives for RZM to provide the flexibility to support variable key/value sizes and value calculations. (3) We provide the hardware design to implement the proposed primitives using only O(Vlog(V)) resource. With the above techniques, a flexible and efficient solution for MSOs has been built. Our design can be used either as a drop-in replacement of the merge unit in prior accelerators to reduce the cost from O(V2) to O(Vlog(V)), or as an extension to the SIMD ISA of CPU and GPU. In our evaluation on CPU, when V=16 (512-bit SIMD, 32-bit element), we achieve significant speedup on a range of representative kernels including set operations (8.4×), database joins (7.3×), sparse vector/matrix/tensor addition/multiplication on real/complex numbers (6.5×), merge sort (8.0× over scalar, 3.4× over the state-of-the-art SIMD), and SpGEMM (4.4× over the best one in the baseline collection). @InProceedings{ASPLOS22p669, author = {Bangyan Wang and Lei Deng and Fei Sun and Guohao Dai and Liu Liu and Yu Wang and Yuan Xie}, title = {A One-for-All and <i>O</i>(<i>V</i> log(<i>V</i> ))-Cost Solution for Parallel Merge Style Operations on Sorted Key-Value Arrays}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {669--682}, doi = {10.1145/3503222.3507728}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Wang, Bolin	ASPLOS '22: "Randomized Row-Swap: Mitigating ..." Randomized Row-Swap: Mitigating Row Hammer by Breaking Spatial Correlation between Aggressor and Victim Rows Gururaj Saileshwar, Bolin Wang, Moinuddin Qureshi, and Prashant J. Nair (Georgia Institute of Technology, USA; University of British Columbia, Canada) Row Hammer is a fault-injection attack in which rapid activations to a single DRAM row causes bit-flips in nearby rows. Several recent defenses propose tracking aggressor-rows and applying mitigating action on neighboring victim rows by refreshing them. However, all such proposals using victim-focused mitigation preserve the spatial connection between victim and aggressor rows. Therefore, these proposals are susceptible to access patterns causing bit-flips in rows beyond the immediate neighbor. For example, the Half-Double attack causes bit-flips in the presence of victim-focused mitigation. We propose Randomized Row-Swap (RRS), a novel mitigation action that breaks the spatial connection between the aggressor and victim DRAM rows. This enables RRS to provide robust defense against even complex Row Hammer access patterns. RRS is an aggressor-focused mitigation that periodically swaps aggressor-rows with other randomly selected rows in memory. This limits the possible damage in any one locality within the DRAM memory. While RRS can be used with any tracking mechanism, we implement it with a Misra-Gries tracker and target a Row Hammer Threshold of 4.8K activations (similar to the state-of-the-art attacks). Our evaluations show that RRS has negligible slowdown (0.4% on average) and provides strong security guarantees for avoiding Row Hammer bit flips even under several years of continuous attack. @InProceedings{ASPLOS22p1056, author = {Gururaj Saileshwar and Bolin Wang and Moinuddin Qureshi and Prashant J. Nair}, title = {Randomized Row-Swap: Mitigating Row Hammer by Breaking Spatial Correlation between Aggressor and Victim Rows}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1056--1069}, doi = {10.1145/3503222.3507716}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Wang, Hao	ASPLOS '22: "TMO: Transparent Memory Offloading ..." TMO: Transparent Memory Offloading in Datacenters Johannes Weiner, Niket Agarwal, Dan Schatzberg, Leon Yang, Hao Wang, Blaise Sanouillet, Bikash Sharma, Tejun Heo, Mayank Jain, Chunqiang Tang, and Dimitrios Skarlatos (Meta, USA; Carnegie Mellon University, USA) The unrelenting growth of the memory needs of emerging datacenter applications, along with ever increasing cost and volatility of DRAM prices, has led to DRAM being a major infrastructure expense. Alternative technologies, such as NVMe SSDs and upcoming NVM devices, offer higher capacity than DRAM at a fraction of the cost and power. One promising approach is to transparently offload colder memory to cheaper memory technologies via kernel or hypervisor techniques. The key challenge, however, is to develop a datacenter-scale solution that is robust in dealing with diverse workloads and large performance variance of different offload devices such as compressed memory, SSD, and NVM. This paper presents TMO, Meta’s transparent memory offloading solution for heterogeneous datacenter environments. TMO introduces a new Linux kernel mechanism that directly measures in realtime the lost work due to resource shortage across CPU, memory, and I/O. Guided by this information and without any prior application knowledge, TMO automatically adjusts how much memory to offload to heterogeneous devices (e.g., compressed memory or SSD) according to the device’s performance characteristics and the application’s sensitivity to memory-access slowdown. TMO holistically identifies offloading opportunities from not only the application containers but also the sidecar containers that provide infrastructure-level functions. To maximize memory savings, TMO targets both anonymous memory and file cache, and balances the swap-in rate of anonymous memory and the reload rate of file pages that were recently evicted from the file cache. TMO has been running in production for more than a year, and has saved between 20-32% of the total memory across millions of servers in our large datacenter fleet. We have successfully upstreamed TMO into the Linux kernel. @InProceedings{ASPLOS22p609, author = {Johannes Weiner and Niket Agarwal and Dan Schatzberg and Leon Yang and Hao Wang and Blaise Sanouillet and Bikash Sharma and Tejun Heo and Mayank Jain and Chunqiang Tang and Dimitrios Skarlatos}, title = {TMO: Transparent Memory Offloading in Datacenters}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {609--621}, doi = {10.1145/3503222.3507731}, year = {2022}, } Publisher's Version
	Wang, Jiyuan	ASPLOS '22: "HeteroGen: Transpiling C to ..." HeteroGen: Transpiling C to Heterogeneous HLS Code with Automated Test Generation and Program Repair Qian Zhang, Jiyuan Wang, Guoqing Harry Xu, and Miryung Kim (University of California at Los Angeles, USA) Despite the trend of incorporating heterogeneity and specialization in hardware, the development of heterogeneous applications is limited to a handful of engineers with deep hardware expertise. We propose HeteroGen that takes C/C++ code as input and automatically generates an HLS version with test behavior preservation and better performance. Key to the success of HeteroGen is adapting the idea of search-based program repair to the heterogeneous computing domain, while addressing two technical challenges. First, the turn-around time of HLS compilation and simulation is much longer than the usual C/C++ compilation and execution time; therefore, HeteroGen applies pattern-oriented program edits guided by common fix patterns and their dependences. Second, behavior and performance checking requires testing, but test cases are often unavailable. Thus, HeteroGen auto-generates test inputs suitable for checking C to HLS-C conversion errors, while providing high branch coverage for the original C code. An evaluation of HeteroGen shows that it produces an HLS-compatible version for nine out of ten real-world heterogeneous applications fully automatically, applying up to 438 lines of edits to produce an HLS version 1.63x faster than the original version. @InProceedings{ASPLOS22p1017, author = {Qian Zhang and Jiyuan Wang and Guoqing Harry Xu and Miryung Kim}, title = {HeteroGen: Transpiling C to Heterogeneous HLS Code with Automated Test Generation and Program Repair}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1017--1029}, doi = {10.1145/3503222.3507748}, year = {2022}, } Publisher's Version Published Artifact Info Artifacts Available Artifacts Functional Results Reproduced
	Wang, Ruoyu	ASPLOS '22: "ViK: Practical Mitigation ..." ViK: Practical Mitigation of Temporal Memory Safety Violations through Object ID Inspection Haehyun Cho, Jinbum Park, Adam Oest, Tiffany Bao, Ruoyu Wang, Yan Shoshitaishvili, Adam Doupé, and Gail-Joon Ahn (Soongsil University, South Korea; Samsung Research, South Korea; PayPal, USA; Arizona State University, USA; Samsung Research, USA) Temporal memory safety violations, such as use-after-free (UAF) vulnerabilities, are a critical security issue for software written in memory-unsafe languages such as C and C++. In this paper, we introduce ViK, a novel, lightweight, and widely applicable runtime defense that can protect both operating system (OS) kernels and user-space applications against temporal memory safety violations. ViK performs object ID inspection, where it assigns a random identifier to every allocated object and stores the identifier in the unused bits of the corresponding pointer. When the pointer is used, ViK inspects the value of a pointer before dereferencing, ensuring that the pointer still references the original object. To the best of our knowledge, this is the first mitigation against temporal memory safety violations that scales to OS kernels. We evaluated the software prototype of ViK on Android and Linux kernels and observed runtime overhead of around 20%. Also, we evaluated a hardware-assisted prototype of ViK on Android kernel, where the runtime overhead was as low as 2%. @InProceedings{ASPLOS22p271, author = {Haehyun Cho and Jinbum Park and Adam Oest and Tiffany Bao and Ruoyu Wang and Yan Shoshitaishvili and Adam Doupé and Gail-Joon Ahn}, title = {ViK: Practical Mitigation of Temporal Memory Safety Violations through Object ID Inspection}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {271--284}, doi = {10.1145/3503222.3507780}, year = {2022}, } Publisher's Version Artifacts Functional
	Wang, Sen	ASPLOS '22: "NASPipe: High Performance ..." NASPipe: High Performance and Reproducible Pipeline Parallel Supernet Training via Causal Synchronous Parallelism Shixiong Zhao, Fanxin Li, Xusheng Chen, Tianxiang Shen, Li Chen, Sen Wang, Nicholas Zhang, Cheng Li, and Heming Cui (University of Hong Kong, China; Huawei Technologies, China; University of Science and Technology of China, China) Supernet training, a prevalent and important paradigm in Neural Architecture Search, embeds the whole DNN architecture search space into one monolithic supernet, iteratively activates a subset of the supernet (i.e., a subnet) for fitting each batch of data, and searches a high-quality subnet which meets specific requirements. Although training subnets in parallel on multiple GPUs is desirable for acceleration, there inherently exists a race hazard that concurrent subnets may access the same DNN layers. Existing systems support neither efficiently parallelizing subnets’ training executions, nor resolving the race hazard deterministically, leading to unreproducible training procedures and potentiallly non-trivial accuracy loss. We present NASPipe, the first high-performance and reproducible distributed supernet training system via causal synchronous parallel (CSP) pipeline scheduling abstraction: NASPipe partitions a supernet across GPUs and concurrently executes multiple generated sub-tasks (subnets) in a pipelined manner; meanwhile, it oversees the correlations between the subnets and deterministically resolves any causal dependency caused by subnets’ layer sharing. To obtain high performance, NASPipe’s CSP scheduler exploits the fact that the larger a supernet spans, the fewer dependencies manifest between chronologically close subnets; therefore, it aggressively schedules the subnets with larger chronological orders into execution, only if they are not causally dependent on unfinished precedent subnets. Moreover, to relieve the excessive GPU memory burden for holding the whole supernet’s parameters, NASPipe uses a context switch technique that stashes the whole supernet in CPU memory, precisely predicts the subnets’ schedule, and pre-fetches/evicts a subnet before/after its execution. The evaluation shows that NASPipe is the only system that retains supernet training reproducibility, while achieving a comparable and even higher performance (up to 7.8X) compared to three recent pipeline training systems (e.g., GPipe). @InProceedings{ASPLOS22p374, author = {Shixiong Zhao and Fanxin Li and Xusheng Chen and Tianxiang Shen and Li Chen and Sen Wang and Nicholas Zhang and Cheng Li and Heming Cui}, title = {NASPipe: High Performance and Reproducible Pipeline Parallel Supernet Training via Causal Synchronous Parallelism}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {374--387}, doi = {10.1145/3503222.3507735}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Wang, Yawen	ASPLOS '22: "SOL: Safe On-Node Learning ..." SOL: Safe On-Node Learning in Cloud Platforms Yawen Wang, Daniel Crankshaw, Neeraja J. Yadwadkar, Daniel Berger, Christos Kozyrakis, and Ricardo Bianchini (Stanford University, USA; Microsoft Research, USA; University of Texas at Austin, USA) Cloud platforms run many software agents on each server node. These agents manage all aspects of node operation, and in some cases frequently collect data and make decisions. Unfortunately, their behavior is typically based on pre-defined static heuristics or offline analysis; they do not leverage on-node machine learning (ML). In this paper, we first characterize the spectrum of node agents in Azure, and identify the classes of agents that are most likely to benefit from on-node ML. We then propose SOL, an extensible framework for designing ML-based agents that are safe and robust to the range of failure conditions that occur in production. SOL provides a simple API to agent developers and manages the scheduling and running of the agent-specific functions they write. We illustrate the use of SOL by implementing three ML-based agents that manage CPU cores, node power, and memory placement. Our experiments show that (1) ML substantially improves our agents, and (2) SOL ensures that agents operate safely under a variety of failure conditions. We conclude that ML-based agents show significant potential and that SOL can help build them. @InProceedings{ASPLOS22p622, author = {Yawen Wang and Daniel Crankshaw and Neeraja J. Yadwadkar and Daniel Berger and Christos Kozyrakis and Ricardo Bianchini}, title = {SOL: Safe On-Node Learning in Cloud Platforms}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {622--634}, doi = {10.1145/3503222.3507704}, year = {2022}, } Publisher's Version
	Wang, Yishuo	ASPLOS '22: "CoolEdge: Hotspot-Relievable ..." CoolEdge: Hotspot-Relievable Warm Water Cooling for Energy-Efficient Edge Datacenters Qiangyu Pei, Shutong Chen, Qixia Zhang, Xinhui Zhu, Fangming Liu, Ziyang Jia, Yishuo Wang, and Yongjie Yuan (Huazhong University of Science and Technology, China) As the computing frontier drifts to the edge, edge datacenters play a crucial role in supporting various real-time applications. Different from cloud datacenters, the requirements of proximity to end-users, high density, and heterogeneity, present new challenges to cool the edge datacenters efficiently. Although warm water cooling has become a promising cooling technique for this infrastructure, the one-size-fits-all cooling control would lower the cooling efficiency considerably because of the severe thermal imbalance across servers, hardware, and even inside one hardware component in an edge datacenter. In this work, we propose CoolEdge, a hotspot-relievable warm water cooling system for improving the cooling efficiency and saving costs of edge datacenters. Specifically, through the elaborate design of water circulations, CoolEdge can dynamically adjust the water temperature and flow rate for each heterogeneous hardware component to eliminate the hardware-level hotspots. By redesigning cold plates, CoolEdge can quickly disperse the chip-level hotspots without manual intervention. We further quantify the power saving achieved by the warm water cooling theoretically, and propose a custom-designed cooling solution to decide an appropriate water temperature and flow rate periodically. Based on a hardware prototype and real-world traces from SURFsara, the evaluation results show that CoolEdge reduces the cooling energy by 81.81% and 71.92%, respectively, compared with conventional and state-of-the-art water cooling systems. @InProceedings{ASPLOS22p814, author = {Qiangyu Pei and Shutong Chen and Qixia Zhang and Xinhui Zhu and Fangming Liu and Ziyang Jia and Yishuo Wang and Yongjie Yuan}, title = {CoolEdge: Hotspot-Relievable Warm Water Cooling for Energy-Efficient Edge Datacenters}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {814--829}, doi = {10.1145/3503222.3507713}, year = {2022}, } Publisher's Version
	Wang, Yu	ASPLOS '22: "A One-for-All and O(V ..." A One-for-All and O(V log(V ))-Cost Solution for Parallel Merge Style Operations on Sorted Key-Value Arrays Bangyan Wang, Lei Deng, Fei Sun, Guohao Dai, Liu Liu, Yu Wang, and Yuan Xie (University of California at Santa Barbara, USA; Tsinghua University, China; Alibaba DAMO Academy, China) The processing of sorted key-value arrays using a “merge style operation (MSO)” is a very basic and important problem in domains like scientific computing, deep learning, database, graph analysis, sorting, set-operation etc. MSOs dominate the execution time in some important applications like SpGEMM and graph mining. For example, sparse vector addition as an MSO takes up to 98% execution time in SpGEMM in our experiment. For this reason, accelerating MSOs on CPU, GPU, and accelerators using parallel execution has been extensively studied but the solutions in prior work have three major limitations. (1) They treat different MSOs as isolated problems using incompatible methods and an unified solution is still lacking. (2) They do not have the flexibility to support variable key/value sizes and value calculations in the runtime given a fixed hardware design. (3) They require a quadratic hardware cost (O(V2)) for given parallelism V in most cases. To address above three limitations, we make the following efforts. (1) We present a one-for-all solution to support all interested MSOs based on a unified abstraction model “restricted zip machine (RZM)”. (2) We propose a set of composable and parallel primitives for RZM to provide the flexibility to support variable key/value sizes and value calculations. (3) We provide the hardware design to implement the proposed primitives using only O(Vlog(V)) resource. With the above techniques, a flexible and efficient solution for MSOs has been built. Our design can be used either as a drop-in replacement of the merge unit in prior accelerators to reduce the cost from O(V2) to O(Vlog(V)), or as an extension to the SIMD ISA of CPU and GPU. In our evaluation on CPU, when V=16 (512-bit SIMD, 32-bit element), we achieve significant speedup on a range of representative kernels including set operations (8.4×), database joins (7.3×), sparse vector/matrix/tensor addition/multiplication on real/complex numbers (6.5×), merge sort (8.0× over scalar, 3.4× over the state-of-the-art SIMD), and SpGEMM (4.4× over the best one in the baseline collection). @InProceedings{ASPLOS22p669, author = {Bangyan Wang and Lei Deng and Fei Sun and Guohao Dai and Liu Liu and Yu Wang and Yuan Xie}, title = {A One-for-All and <i>O</i>(<i>V</i> log(<i>V</i> ))-Cost Solution for Parallel Merge Style Operations on Sorted Key-Value Arrays}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {669--682}, doi = {10.1145/3503222.3507728}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Weimer, Westley	ASPLOS '22: "CirFix: Automatically Repairing ..." CirFix: Automatically Repairing Defects in Hardware Design Code Hammad Ahmad, Yu Huang, and Westley Weimer (University of Michigan, USA) This paper presents CirFix, a framework for automatically repairing defects in hardware designs implemented in languages like Verilog. We propose a novel fault localization approach based on assignments to wires and registers, and a fitness function tailored to the hardware domain to bridge the gap between software-level automated program repair and hardware descriptions. We also present a benchmark suite of 32 defect scenarios corresponding to a variety of hardware projects. Overall, CirFix produces plausible repairs for 21/32 and correct repairs for 16/32 of the defect scenarios. This repair rate is comparable to that of successful program repair approaches for software, indicating CirFix is effective at bringing over the benefits of automated program repair to the hardware domain for the first time. @InProceedings{ASPLOS22p990, author = {Hammad Ahmad and Yu Huang and Westley Weimer}, title = {CirFix: Automatically Repairing Defects in Hardware Design Code}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {990--1003}, doi = {10.1145/3503222.3507763}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Weiner, Johannes	ASPLOS '22: "TMO: Transparent Memory Offloading ..." TMO: Transparent Memory Offloading in Datacenters Johannes Weiner, Niket Agarwal, Dan Schatzberg, Leon Yang, Hao Wang, Blaise Sanouillet, Bikash Sharma, Tejun Heo, Mayank Jain, Chunqiang Tang, and Dimitrios Skarlatos (Meta, USA; Carnegie Mellon University, USA) The unrelenting growth of the memory needs of emerging datacenter applications, along with ever increasing cost and volatility of DRAM prices, has led to DRAM being a major infrastructure expense. Alternative technologies, such as NVMe SSDs and upcoming NVM devices, offer higher capacity than DRAM at a fraction of the cost and power. One promising approach is to transparently offload colder memory to cheaper memory technologies via kernel or hypervisor techniques. The key challenge, however, is to develop a datacenter-scale solution that is robust in dealing with diverse workloads and large performance variance of different offload devices such as compressed memory, SSD, and NVM. This paper presents TMO, Meta’s transparent memory offloading solution for heterogeneous datacenter environments. TMO introduces a new Linux kernel mechanism that directly measures in realtime the lost work due to resource shortage across CPU, memory, and I/O. Guided by this information and without any prior application knowledge, TMO automatically adjusts how much memory to offload to heterogeneous devices (e.g., compressed memory or SSD) according to the device’s performance characteristics and the application’s sensitivity to memory-access slowdown. TMO holistically identifies offloading opportunities from not only the application containers but also the sidecar containers that provide infrastructure-level functions. To maximize memory savings, TMO targets both anonymous memory and file cache, and balances the swap-in rate of anonymous memory and the reload rate of file pages that were recently evicted from the file cache. TMO has been running in production for more than a year, and has saved between 20-32% of the total memory across millions of servers in our large datacenter fleet. We have successfully upstreamed TMO into the Linux kernel. @InProceedings{ASPLOS22p609, author = {Johannes Weiner and Niket Agarwal and Dan Schatzberg and Leon Yang and Hao Wang and Blaise Sanouillet and Bikash Sharma and Tejun Heo and Mayank Jain and Chunqiang Tang and Dimitrios Skarlatos}, title = {TMO: Transparent Memory Offloading in Datacenters}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {609--621}, doi = {10.1145/3503222.3507731}, year = {2022}, } Publisher's Version
	Weinhold, Carsten	ASPLOS '22: "Efficient and Scalable Core ..." Efficient and Scalable Core Multiplexing with M³v Nils Asmussen, Sebastian Haas, Carsten Weinhold, Till Miemietz, and Michael Roitzsch (Barkhausen Institut, Germany) The M³ system (ASPLOS ’16) proposed a hardware/software co-design that simplifies integration between general-purpose cores and special-purpose accelerators, allowing users to easily utilize them in a unified manner. M³ is a tiled architecture, whose tiles (cores and accelerators) are partitioned between applications, such that each tile is dedicated to its own application. The M³x system (ATC ’19) extended M³ by trading off some isolation to enable coarse-grained multiplexing of tiles among multiple applications. With M³x, if source tile t₁ runs code of application p and sends a message m to destination tile t₂ while t₂ is currently not associated with p, then m is forwarded to the right place through a “slow path”, via some special OS tile. In this paper, we present M³v, which extends M³x by further trading off some isolation between applications to support “fast path” communication that does not require the said OS tile’s involvement. Thus, with M³v, a tile can be efficiently multiplexed between applications provided it is a general-purpose core. M³v achieves this goal by 1) adding a local multiplexer to each such core, and by 2) virtualizing the core’s hardware component responsible for cross-tile communications. We prototype M³v using RISC-V cores on an FPGA platform and show that it significantly outperforms M³x and may achieve competitive performance to Linux. @InProceedings{ASPLOS22p452, author = {Nils Asmussen and Sebastian Haas and Carsten Weinhold and Till Miemietz and Michael Roitzsch}, title = {Efficient and Scalable Core Multiplexing with M³v}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {452--466}, doi = {10.1145/3503222.3507741}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Wijerathne, Dhananjaya	ASPLOS '22: "REVAMP: A Systematic Framework ..." REVAMP: A Systematic Framework for Heterogeneous CGRA Realization Thilini Kaushalya Bandara, Dhananjaya Wijerathne, Tulika Mitra, and Li-Shiuan Peh (National University of Singapore, Singapore) Coarse-Grained Reconfigurable Architectures (CGRAs) provide an excellent balance between performance, energy efficiency, and flexibility. However, increasingly sophisticated applications, especially on the edge devices, demand even better energy efficiency for longer battery life. Most CGRAs adhere to a canonical structure where a homogeneous set of processing elements and memories communicate through a regular interconnect due to the simplicity of the design. Unfortunately, the homogeneity leads to substantial idle resources while mapping irregular applications and creates inefficiency. We plan to mitigate the inefficiency by systematically and judiciously introducing heterogeneity in CGRAs in tandem with appropriate compiler support. We propose REVAMP, an automated design space exploration framework that helps architects uncover and add pertinent heterogeneity to a diverse range of originally homogeneous CGRAs when fed with a suite of target applications. REVAMP explores a comprehensive set of optimizations encompassing compute, network, and memory heterogeneity, thereby converting a uniform CGRA into a more irregular architecture with improved energy efficiency. As CGRAs are inherently software scheduled, any micro-architectural optimizations need to be partnered with corresponding compiler support, which is challenging with heterogeneity. The REVAMP framework extends compiler support for efficient mapping of loop kernels on the derived heterogeneous CGRA architectures. We showcase REVAMP on three state-of-the-art homogeneous CGRAs, demonstrating how REVAMP derives a heterogeneous variant of each homogeneous architecture, with its corresponding compiler optimizations. Our results show that the derived heterogeneous architectures achieve up to 52.4% power reduction, 38.1% area reduction, and 36% average energy reduction over the corresponding homogeneous versions with minimal performance impact for the selected kernel suite. @InProceedings{ASPLOS22p918, author = {Thilini Kaushalya Bandara and Dhananjaya Wijerathne and Tulika Mitra and Li-Shiuan Peh}, title = {REVAMP: A Systematic Framework for Heterogeneous CGRA Realization}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {918--932}, doi = {10.1145/3503222.3507772}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Wills, Lisa Wu	ASPLOS '22: "ProSE: The Architecture and ..." ProSE: The Architecture and Design of a Protein Discovery Engine Eyes Robson, Ceyu Xu, and Lisa Wu Wills (University of California at Berkeley, USA; Duke University, USA) Protein language models have enabled breakthrough approaches to protein structure prediction, function annotation, and drug discovery. A primary limitation to the widespread adoption of these powerful models is the high computational cost associated with the training and inference of these models, especially at longer sequence lengths. We present the architecture, microarchitecture, and hardware implementation of a protein design and discovery accelerator, ProSE (Protein Systolic Engine). ProSE has a collection of custom heterogeneous systolic arrays and special functions that process transfer learning model inferences efficiently. The architecture marries SIMD-style computations with systolic array architectures, optimizing coarse-grained operation sequences across model layers to achieve efficiency without sacrificing generality. ProSE performs Protein BERT inference at up to 6.9× speedup and 48× power efficiency (performance/Watt) compared to one NVIDIA A100 GPU. ProSE achieves up to 5.5 × (12.7×) speedup and 173× (249×) power efficiency compared to TPUv3 (TPUv2). @InProceedings{ASPLOS22p655, author = {Eyes Robson and Ceyu Xu and Lisa Wu Wills}, title = {ProSE: The Architecture and Design of a Protein Discovery Engine}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {655--668}, doi = {10.1145/3503222.3507722}, year = {2022}, } Publisher's Version
	Wu, Anbang	ASPLOS '22: "Paulihedral: A Generalized ..." Paulihedral: A Generalized Block-Wise Compiler Optimization Framework for Quantum Simulation Kernels Gushu Li, Anbang Wu, Yunong Shi, Ali Javadi-Abhari, Yufei Ding, and Yuan Xie (University of California at Santa Barbara, USA; Amazon, USA; IBM, USA) The quantum simulation kernel is an important subroutine appearing as a very long gate sequence in many quantum programs. In this paper, we propose Paulihedral, a block-wise compiler framework that can deeply optimize this subroutine by exploiting high-level program structure and optimization opportunities. Paulihedral first employs a new Pauli intermediate representation that can maintain the high-level semantics and constraints in quantum simulation kernels. This naturally enables new large-scale optimizations that are hard to implement at the low gate-level. In particular, we propose two technology-independent instruction scheduling passes, and two technology-dependent code optimization passes which reconcile the circuit synthesis, gate cancellation, and qubit mapping stages of the compiler. Experimental results show that Paulihedral can outperform state-of-the-art compiler infrastructures in a wide-range of applications on both near-term superconducting quantum processors and future fault-tolerant quantum computers. @InProceedings{ASPLOS22p554, author = {Gushu Li and Anbang Wu and Yunong Shi and Ali Javadi-Abhari and Yufei Ding and Yuan Xie}, title = {Paulihedral: A Generalized Block-Wise Compiler Optimization Framework for Quantum Simulation Kernels}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {554--569}, doi = {10.1145/3503222.3507715}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Wu, Carole-Jean	ASPLOS '22: "RecShard: Statistical Feature-Based ..." RecShard: Statistical Feature-Based Memory Optimization for Industry-Scale Neural Recommendation Geet Sethi, Bilge Acun, Niket Agarwal, Christos Kozyrakis, Caroline Trippel, and Carole-Jean Wu (Stanford University, USA; Meta, USA) We propose RecShard, a fine-grained embedding table (EMB) partitioning and placement technique for deep learning recommendation models (DLRMs). RecShard is designed based on two key observations. First, not all EMBs are equal, nor all rows within an EMB are equal in terms of access patterns. EMBs exhibit distinct memory characteristics, providing performance optimization opportunities for intelligent EMB partitioning and placement across a tiered memory hierarchy. Second, in modern DLRMs, EMBs function as hash tables. As a result, EMBs display interesting phenomena, such as the birthday paradox, leaving EMBs severely under-utilized. RecShard determines an optimal EMB sharding strategy for a set of EMBs based on training data distributions and model characteristics, along with the bandwidth characteristics of the underlying tiered memory hierarchy. In doing so, RecShard achieves over 6 times higher EMB training throughput on average for capacity constrained DLRMs. The throughput increase comes from improved EMB load balance by over 12 times and from the reduced access to the slower memory by over 87 times. @InProceedings{ASPLOS22p344, author = {Geet Sethi and Bilge Acun and Niket Agarwal and Christos Kozyrakis and Caroline Trippel and Carole-Jean Wu}, title = {RecShard: Statistical Feature-Based Memory Optimization for Industry-Scale Neural Recommendation}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {344--358}, doi = {10.1145/3503222.3507777}, year = {2022}, } Publisher's Version
	Wu, Lei	ASPLOS '22: "EXAMINER: Automatically Locating ..." EXAMINER: Automatically Locating Inconsistent Instructions between Real Devices and CPU Emulators for ARM Muhui Jiang, Tianyi Xu, Yajin Zhou, Yufeng Hu, Ming Zhong, Lei Wu, Xiapu Luo, and Kui Ren (Hong Kong Polytechnic University, China; Zhejiang University, China) Emulators are widely used to build dynamic analysis frameworks due to its fine-grained tracing capability, full system monitoring functionality, and scalability of running on different operating systems and architectures. However, whether emulators are consistent with real devices is unknown. To understand this problem, we aim to automatically locate inconsistent instructions, which behave differently between emulators and real devices. We target the ARM architecture, which provides machine-readable specifications. Based on the specification, we propose a sufficient test case generator by designing and implementing the first symbolic execution engine for the ARM architecture specification language (ASL). We generate 2,774,649 representative instruction streams and conduct differential testing between four ARM real devices in different architecture versions (i.e., ARMv5, ARMv6, ARMv7, and ARMv8) and three state-of-the-art emulators (i.e., QEMU, Unicorn, and Angr). We locate a huge number of inconsistent instruction streams (171,858 for QEMU, 223,264 for unicorn, and 120,169 for Angr). We find that undefined implementation in ARM manual and bugs of emulators are the major causes of inconsistencies. Furthermore, we discover 12 bugs, which influence commonly used instructions (e.g., BLX). With the inconsistent instructions, we build three security applications and demonstrate the capability of these instructions on detecting emulators, anti-emulation, and anti-fuzzing. @InProceedings{ASPLOS22p846, author = {Muhui Jiang and Tianyi Xu and Yajin Zhou and Yufeng Hu and Ming Zhong and Lei Wu and Xiapu Luo and Kui Ren}, title = {EXAMINER: Automatically Locating Inconsistent Instructions between Real Devices and CPU Emulators for ARM}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {846--858}, doi = {10.1145/3503222.3507736}, year = {2022}, } Publisher's Version
	Xia, Shihao	ASPLOS '22: "Who Goes First? Detecting ..." Who Goes First? Detecting Go Concurrency Bugs via Message Reordering Ziheng Liu, Shihao Xia, Yu Liang, Linhai Song, and Hong Hu (Pennsylvania State University, USA) Go is a young programming language invented to build safe and efficient concurrent programs. It provides goroutines as lightweight threads and channels for inter-goroutine communication. Programmers are encouraged to explicitly pass messages through channels to connect goroutines, with the purpose of reducing the chance of making programming mistakes and introducing concurrency bugs. Go is one of the most beloved programming languages and has already been used to build many critical infrastructure software systems in the data-center environment. However, a recent study shows that channel-related concurrency bugs are still common in Go programs, severely hurting the reliability of the programs. This paper presents GFuzz, a dynamic detector that can effectively pinpoint channel-related concurrency bugs by mutating the processing orders of concurrent messages. We build GFuzz in three steps. We first adopt an effective approach to identify concurrent messages and transform a program to process those messages in any given order. We then take a fuzzing approach to generate new processing orders by mutating exercised ones and rely on execution feedback to prioritize orders close to triggering bugs. Finally, we design a runtime sanitizer to capture triggered bugs that are missed by the Go runtime. We evaluate GFuzz on seven popular Go software systems, including Docker, Kubernetes, and gRPC. GFuzz finds 184 previously unknown bugs and reports a negligible number of false positives. Programmers have already confirmed 124 reports as real bugs and fixed 67 of them based on our reporting. A careful inspection of the detected concurrency bugs from gRPC shows the effectiveness of each component of GFuzz and confirms the components' rationality. @InProceedings{ASPLOS22p888, author = {Ziheng Liu and Shihao Xia and Yu Liang and Linhai Song and Hong Hu}, title = {Who Goes First? Detecting Go Concurrency Bugs via Message Reordering}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {888--902}, doi = {10.1145/3503222.3507753}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional
	Xia, Yubin	ASPLOS '22: "Serverless Computing on Heterogeneous ..." Serverless Computing on Heterogeneous Computers Dong Du, Qingyuan Liu, Xueqiang Jiang, Yubin Xia, Binyu Zang, and Haibo Chen (Shanghai Jiao Tong University, China; Shanghai Artificial Intelligence Laboratory, China) Existing serverless computing platforms are built upon homogeneous computers, limiting the function density and restricting serverless computing to limited scenarios. We introduce Molecule, the first serverless computing system utilizing heterogeneous computers. Molecule enables both general-purpose devices (e.g., Nvidia DPU) and domain-specific accelerators (e.g., FPGA and GPU) for serverless applications that significantly improve function density (50% higher) and application performance (up to 34.6x). To achieve these results, we first propose XPU-Shim, a distributed shim to bridge the gap between underlying multi-OS systems (when using general-purpose devices) and our serverless runtime (i.e., Molecule). We further introduce vectorized sandbox, a sandbox abstraction to abstract hardware heterogeneity (when using domain-specific accelerators). Moreover, we also review state-of-the-art serverless optimizations on startup and communication latency and overcome the challenges to implement them on heterogeneous computers. We have implemented Molecule on real platforms with Nvidia DPUs and Xilinx FPGAs and evaluate it using benchmarks and real-world applications. @InProceedings{ASPLOS22p797, author = {Dong Du and Qingyuan Liu and Xueqiang Jiang and Yubin Xia and Binyu Zang and Haibo Chen}, title = {Serverless Computing on Heterogeneous Computers}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {797--813}, doi = {10.1145/3503222.3507732}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Xiao, Yuanlong	ASPLOS '22: "PLD: Fast FPGA Compilation ..." PLD: Fast FPGA Compilation to Make Reconfigurable Acceleration Compatible with Modern Incremental Refinement Software Development Yuanlong Xiao, Eric Micallef, Andrew Butt, Matthew Hofmann, Marc Alston, Matthew Goldsmith, Andrew Merczynski-Hait, and André DeHon (University of Pennsylvania, USA) FPGA-based accelerators are demonstrating significant absolute performance and energy efficiency compared with general-purpose CPUs. While FPGA computations can now be described in standard, programming languages, like C, development for FPGAs accelerators remains tedious and inaccessible to modern software engineers. Slow compiles (potentially taking tens of hours) inhibit the rapid, incremental refinement of designs that is the hallmark of modern software engineering. To address this issue, we introduce separate compilation and linkage into the FPGA design flow, providing faster design turns more familiar to software development. To realize this flow, we provide abstractions, compiler options, and compiler flow that allow the same C source code to be compiled to processor cores in seconds and to FPGA regions in minutes, providing the missing -O0 and -O1 options familiar in software development. This raises the FPGA programming level and standardizes the programming experience, bringing FPGA-based accelerators into a more familiar software platform ecosystem for software engineers. @InProceedings{ASPLOS22p933, author = {Yuanlong Xiao and Eric Micallef and Andrew Butt and Matthew Hofmann and Marc Alston and Matthew Goldsmith and Andrew Merczynski-Hait and André DeHon}, title = {PLD: Fast FPGA Compilation to Make Reconfigurable Acceleration Compatible with Modern Incremental Refinement Software Development}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {933--945}, doi = {10.1145/3503222.3507740}, year = {2022}, } Publisher's Version
	Xie, Lei	ASPLOS '22: "Suppressing ZZ Crosstalk of ..." Suppressing ZZ Crosstalk of Quantum Computers through Pulse and Scheduling Co-Optimization Lei Xie, Jidong Zhai, ZhenXing Zhang, Jonathan Allcock, Shengyu Zhang, and Yi-Cong Zheng (Tsinghua University, China; Tencent Quantum Laboratory, China) Noise is a significant obstacle to quantum computing, and ZZ crosstalk is one of the most destructive types of noise affecting superconducting qubits. Previous approaches to suppressing ZZ crosstalk have mainly relied on specific chip design that can complicate chip fabrication and aggravate decoherence. To some extent, special chip design can be avoided by relying on pulse optimization to suppress ZZ crosstalk. However, existing approaches are non-scalable, as their required time and memory grow exponentially with the number of qubits involved. To address the above problems, we propose a scalable approach by co-optimizing pulses and scheduling. We optimize pulses to offer an ability to suppress ZZ crosstalk surrounding a gate, and then design scheduling strategies to exploit this ability and achieve suppression across the whole circuit. A main advantage of such co-optimization is that it does not require special hardware support. Besides, we implement our approach as a general framework that is compatible with different pulse optimization methods. We have conducted extensive evaluations by simulation and on a real quantum computer. Simulation results show that our proposal can improve the fidelity of quantum computing on 4∼12 qubits by up to 81× (11× on average). Ramsey experiments on a real quantum computer also demonstrate that our method can eliminate the effect of ZZ crosstalk to a great extent. @InProceedings{ASPLOS22p499, author = {Lei Xie and Jidong Zhai and ZhenXing Zhang and Jonathan Allcock and Shengyu Zhang and Yi-Cong Zheng}, title = {Suppressing ZZ Crosstalk of Quantum Computers through Pulse and Scheduling Co-Optimization}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {499--513}, doi = {10.1145/3503222.3507761}, year = {2022}, } Publisher's Version
	Xie, Yuan	ASPLOS '22: "DOTA: Detect and Omit Weak ..." DOTA: Detect and Omit Weak Attentions for Scalable Transformer Acceleration Zheng Qu, Liu Liu, Fengbin Tu, Zhaodong Chen, Yufei Ding, and Yuan Xie (University of California at Santa Barbara, USA) Transformer Neural Networks have demonstrated leading performance in many applications spanning over language understanding, image processing, and generative modeling. Despite the impressive performance, long-sequence Transformer processing is expensive due to quadratic computation complexity and memory consumption of self-attention. In this paper, we present DOTA, an algorithm-architecture co-design that effectively addresses the challenges of scalable Transformer inference. Based on the insight that not all connections in an attention graph are equally important, we propose to jointly optimize a lightweight Detector with the Transformer model to accurately detect and omit weak connections during runtime. Furthermore, we design a specialized system architecture for end-to-end Transformer acceleration using the proposed attention detection mechanism. Experiments on a wide range of benchmarks demonstrate the superior performance of DOTA over other solutions. In summary, DOTA achieves 152.6x and 4.5x performance speedup and orders of magnitude energy-efficiency improvements over GPU and customized hardware, respectively. @InProceedings{ASPLOS22p14, author = {Zheng Qu and Liu Liu and Fengbin Tu and Zhaodong Chen and Yufei Ding and Yuan Xie}, title = {DOTA: Detect and Omit Weak Attentions for Scalable Transformer Acceleration}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {14--26}, doi = {10.1145/3503222.3507738}, year = {2022}, } Publisher's Version ASPLOS '22: "Paulihedral: A Generalized ..." Paulihedral: A Generalized Block-Wise Compiler Optimization Framework for Quantum Simulation Kernels Gushu Li, Anbang Wu, Yunong Shi, Ali Javadi-Abhari, Yufei Ding, and Yuan Xie (University of California at Santa Barbara, USA; Amazon, USA; IBM, USA) The quantum simulation kernel is an important subroutine appearing as a very long gate sequence in many quantum programs. In this paper, we propose Paulihedral, a block-wise compiler framework that can deeply optimize this subroutine by exploiting high-level program structure and optimization opportunities. Paulihedral first employs a new Pauli intermediate representation that can maintain the high-level semantics and constraints in quantum simulation kernels. This naturally enables new large-scale optimizations that are hard to implement at the low gate-level. In particular, we propose two technology-independent instruction scheduling passes, and two technology-dependent code optimization passes which reconcile the circuit synthesis, gate cancellation, and qubit mapping stages of the compiler. Experimental results show that Paulihedral can outperform state-of-the-art compiler infrastructures in a wide-range of applications on both near-term superconducting quantum processors and future fault-tolerant quantum computers. @InProceedings{ASPLOS22p554, author = {Gushu Li and Anbang Wu and Yunong Shi and Ali Javadi-Abhari and Yufei Ding and Yuan Xie}, title = {Paulihedral: A Generalized Block-Wise Compiler Optimization Framework for Quantum Simulation Kernels}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {554--569}, doi = {10.1145/3503222.3507715}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced ASPLOS '22: "A One-for-All and O(V ..." A One-for-All and O(V log(V ))-Cost Solution for Parallel Merge Style Operations on Sorted Key-Value Arrays Bangyan Wang, Lei Deng, Fei Sun, Guohao Dai, Liu Liu, Yu Wang, and Yuan Xie (University of California at Santa Barbara, USA; Tsinghua University, China; Alibaba DAMO Academy, China) The processing of sorted key-value arrays using a “merge style operation (MSO)” is a very basic and important problem in domains like scientific computing, deep learning, database, graph analysis, sorting, set-operation etc. MSOs dominate the execution time in some important applications like SpGEMM and graph mining. For example, sparse vector addition as an MSO takes up to 98% execution time in SpGEMM in our experiment. For this reason, accelerating MSOs on CPU, GPU, and accelerators using parallel execution has been extensively studied but the solutions in prior work have three major limitations. (1) They treat different MSOs as isolated problems using incompatible methods and an unified solution is still lacking. (2) They do not have the flexibility to support variable key/value sizes and value calculations in the runtime given a fixed hardware design. (3) They require a quadratic hardware cost (O(V2)) for given parallelism V in most cases. To address above three limitations, we make the following efforts. (1) We present a one-for-all solution to support all interested MSOs based on a unified abstraction model “restricted zip machine (RZM)”. (2) We propose a set of composable and parallel primitives for RZM to provide the flexibility to support variable key/value sizes and value calculations. (3) We provide the hardware design to implement the proposed primitives using only O(Vlog(V)) resource. With the above techniques, a flexible and efficient solution for MSOs has been built. Our design can be used either as a drop-in replacement of the merge unit in prior accelerators to reduce the cost from O(V2) to O(Vlog(V)), or as an extension to the SIMD ISA of CPU and GPU. In our evaluation on CPU, when V=16 (512-bit SIMD, 32-bit element), we achieve significant speedup on a range of representative kernels including set operations (8.4×), database joins (7.3×), sparse vector/matrix/tensor addition/multiplication on real/complex numbers (6.5×), merge sort (8.0× over scalar, 3.4× over the state-of-the-art SIMD), and SpGEMM (4.4× over the best one in the baseline collection). @InProceedings{ASPLOS22p669, author = {Bangyan Wang and Lei Deng and Fei Sun and Guohao Dai and Liu Liu and Yu Wang and Yuan Xie}, title = {A One-for-All and <i>O</i>(<i>V</i> log(<i>V</i> ))-Cost Solution for Parallel Merge Style Operations on Sorted Key-Value Arrays}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {669--682}, doi = {10.1145/3503222.3507728}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Xu, Ceyu	ASPLOS '22: "ProSE: The Architecture and ..." ProSE: The Architecture and Design of a Protein Discovery Engine Eyes Robson, Ceyu Xu, and Lisa Wu Wills (University of California at Berkeley, USA; Duke University, USA) Protein language models have enabled breakthrough approaches to protein structure prediction, function annotation, and drug discovery. A primary limitation to the widespread adoption of these powerful models is the high computational cost associated with the training and inference of these models, especially at longer sequence lengths. We present the architecture, microarchitecture, and hardware implementation of a protein design and discovery accelerator, ProSE (Protein Systolic Engine). ProSE has a collection of custom heterogeneous systolic arrays and special functions that process transfer learning model inferences efficiently. The architecture marries SIMD-style computations with systolic array architectures, optimizing coarse-grained operation sequences across model layers to achieve efficiency without sacrificing generality. ProSE performs Protein BERT inference at up to 6.9× speedup and 48× power efficiency (performance/Watt) compared to one NVIDIA A100 GPU. ProSE achieves up to 5.5 × (12.7×) speedup and 173× (249×) power efficiency compared to TPUv3 (TPUv2). @InProceedings{ASPLOS22p655, author = {Eyes Robson and Ceyu Xu and Lisa Wu Wills}, title = {ProSE: The Architecture and Design of a Protein Discovery Engine}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {655--668}, doi = {10.1145/3503222.3507722}, year = {2022}, } Publisher's Version
	Xu, Guoqing Harry	ASPLOS '22: "HeteroGen: Transpiling C to ..." HeteroGen: Transpiling C to Heterogeneous HLS Code with Automated Test Generation and Program Repair Qian Zhang, Jiyuan Wang, Guoqing Harry Xu, and Miryung Kim (University of California at Los Angeles, USA) Despite the trend of incorporating heterogeneity and specialization in hardware, the development of heterogeneous applications is limited to a handful of engineers with deep hardware expertise. We propose HeteroGen that takes C/C++ code as input and automatically generates an HLS version with test behavior preservation and better performance. Key to the success of HeteroGen is adapting the idea of search-based program repair to the heterogeneous computing domain, while addressing two technical challenges. First, the turn-around time of HLS compilation and simulation is much longer than the usual C/C++ compilation and execution time; therefore, HeteroGen applies pattern-oriented program edits guided by common fix patterns and their dependences. Second, behavior and performance checking requires testing, but test cases are often unavailable. Thus, HeteroGen auto-generates test inputs suitable for checking C to HLS-C conversion errors, while providing high branch coverage for the original C code. An evaluation of HeteroGen shows that it produces an HLS-compatible version for nine out of ten real-world heterogeneous applications fully automatically, applying up to 438 lines of edits to produce an HLS version 1.63x faster than the original version. @InProceedings{ASPLOS22p1017, author = {Qian Zhang and Jiyuan Wang and Guoqing Harry Xu and Miryung Kim}, title = {HeteroGen: Transpiling C to Heterogeneous HLS Code with Automated Test Generation and Program Repair}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1017--1029}, doi = {10.1145/3503222.3507748}, year = {2022}, } Publisher's Version Published Artifact Info Artifacts Available Artifacts Functional Results Reproduced ASPLOS '22: "Yashme: Detecting Persistency ..." Yashme: Detecting Persistency Races Hamed Gorjiara, Guoqing Harry Xu, and Brian Demsky (University of California at Irvine, USA; University of California at Los Angeles, USA) Persistent memory (PM) or Non-Volatile Random-Access Memory (NVRAM) hardware such as Intel’s Optane memory product promises to transform how programs store and manipulate information. Ensuring that persistent memory programs are crash consistent is a major challenge. We present a novel class of crash consistency bugs for persistent memory programs, which we call persistency races. Persistency races can cause non-atomic stores to be made partially persistent. Persistency races arise due to the interaction of standard compiler optimizations with persistent memory semantics. We present Yashme, the first detector for persistency races. A major challenge is that in order to detect persistency races, the execution must crash in a very narrow window between a store with a persistency race and its corresponding cache flush operation, making it challenging for naïve techniques to be effective. Yashme overcomes this challenge with a novel technique for detecting races in executions that are prefixes of the pre-crash execution. This technique enables Yashme to effectively find persistency races even if the injected crashes do not fall into that window. We have evaluated Yashme on a range of persistent memory benchmarks and have found 26 real persistency races that have never been reported before. @InProceedings{ASPLOS22p830, author = {Hamed Gorjiara and Guoqing Harry Xu and Brian Demsky}, title = {Yashme: Detecting Persistency Races}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {830--845}, doi = {10.1145/3503222.3507766}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Xu, Tianyi	ASPLOS '22: "EXAMINER: Automatically Locating ..." EXAMINER: Automatically Locating Inconsistent Instructions between Real Devices and CPU Emulators for ARM Muhui Jiang, Tianyi Xu, Yajin Zhou, Yufeng Hu, Ming Zhong, Lei Wu, Xiapu Luo, and Kui Ren (Hong Kong Polytechnic University, China; Zhejiang University, China) Emulators are widely used to build dynamic analysis frameworks due to its fine-grained tracing capability, full system monitoring functionality, and scalability of running on different operating systems and architectures. However, whether emulators are consistent with real devices is unknown. To understand this problem, we aim to automatically locate inconsistent instructions, which behave differently between emulators and real devices. We target the ARM architecture, which provides machine-readable specifications. Based on the specification, we propose a sufficient test case generator by designing and implementing the first symbolic execution engine for the ARM architecture specification language (ASL). We generate 2,774,649 representative instruction streams and conduct differential testing between four ARM real devices in different architecture versions (i.e., ARMv5, ARMv6, ARMv7, and ARMv8) and three state-of-the-art emulators (i.e., QEMU, Unicorn, and Angr). We locate a huge number of inconsistent instruction streams (171,858 for QEMU, 223,264 for unicorn, and 120,169 for Angr). We find that undefined implementation in ARM manual and bugs of emulators are the major causes of inconsistencies. Furthermore, we discover 12 bugs, which influence commonly used instructions (e.g., BLX). With the inconsistent instructions, we build three security applications and demonstrate the capability of these instructions on detecting emulators, anti-emulation, and anti-fuzzing. @InProceedings{ASPLOS22p846, author = {Muhui Jiang and Tianyi Xu and Yajin Zhou and Yufeng Hu and Ming Zhong and Lei Wu and Xiapu Luo and Kui Ren}, title = {EXAMINER: Automatically Locating Inconsistent Instructions between Real Devices and CPU Emulators for ARM}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {846--858}, doi = {10.1145/3503222.3507736}, year = {2022}, } Publisher's Version
	Xu, Tianyin	ASPLOS '22: "Parallel Virtualized Memory ..." Parallel Virtualized Memory Translation with Nested Elastic Cuckoo Page Tables Jovan Stojkovic, Dimitrios Skarlatos, Apostolos Kokolis, Tianyin Xu, and Josep Torrellas (University of Illinois at Urbana-Champaign, USA; Carnegie Mellon University, USA) A major reason why nested or virtualized address translations are slow is because current systems organize page tables in a multi-level tree that is accessed in a sequential manner. A nested translation may potentially require up to twenty-four sequential memory accesses. To address this problem, this paper presents the first page table design that supports parallel nested address translation. The design is based on using hashed page tables (HPTs) for both guest and host. However, directly extending a native HPT design to a nested environment leads to minor gains. Instead, our design solves a new set of challenges that appear in nested environments. Our scheme eliminates all but three of the potentially twenty-four sequential steps of a nested translation—while judiciously limiting the number of parallel memory accesses issued to avoid over-consuming cache bandwidth. As a result, compared to conventional nested radix tables, our design speeds-up the execution of a set of applications by an average of 1.19x (for 4KB pages) and 1.24x (when huge pages are used). In addition, we also show a migration path from current nested radix page tables to our design. @InProceedings{ASPLOS22p84, author = {Jovan Stojkovic and Dimitrios Skarlatos and Apostolos Kokolis and Tianyin Xu and Josep Torrellas}, title = {Parallel Virtualized Memory Translation with Nested Elastic Cuckoo Page Tables}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {84--97}, doi = {10.1145/3503222.3507720}, year = {2022}, } Publisher's Version
	Yadwadkar, Neeraja J.	ASPLOS '22: "SOL: Safe On-Node Learning ..." SOL: Safe On-Node Learning in Cloud Platforms Yawen Wang, Daniel Crankshaw, Neeraja J. Yadwadkar, Daniel Berger, Christos Kozyrakis, and Ricardo Bianchini (Stanford University, USA; Microsoft Research, USA; University of Texas at Austin, USA) Cloud platforms run many software agents on each server node. These agents manage all aspects of node operation, and in some cases frequently collect data and make decisions. Unfortunately, their behavior is typically based on pre-defined static heuristics or offline analysis; they do not leverage on-node machine learning (ML). In this paper, we first characterize the spectrum of node agents in Azure, and identify the classes of agents that are most likely to benefit from on-node ML. We then propose SOL, an extensible framework for designing ML-based agents that are safe and robust to the range of failure conditions that occur in production. SOL provides a simple API to agent developers and manages the scheduling and running of the agent-specific functions they write. We illustrate the use of SOL by implementing three ML-based agents that manage CPU cores, node power, and memory placement. Our experiments show that (1) ML substantially improves our agents, and (2) SOL ensures that agents operate safely under a variety of failure conditions. We conclude that ML-based agents show significant potential and that SOL can help build them. @InProceedings{ASPLOS22p622, author = {Yawen Wang and Daniel Crankshaw and Neeraja J. Yadwadkar and Daniel Berger and Christos Kozyrakis and Ricardo Bianchini}, title = {SOL: Safe On-Node Learning in Cloud Platforms}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {622--634}, doi = {10.1145/3503222.3507704}, year = {2022}, } Publisher's Version
	Yan, Mengjia	ASPLOS '22: "DAGguise: Mitigating Memory ..." DAGguise: Mitigating Memory Timing Side Channels Peter W. Deutsch, Yuheng Yang, Thomas Bourgeat, Jules Drean, Joel S. Emer, and Mengjia Yan (Massachusetts Institute of Technology, USA; NVIDIA, USA) This paper studies the mitigation of memory timing side channels, where attackers utilize contention within DRAM controllers to infer a victim’s secrets. Already practical, this class of channels poses an important challenge to secure computing in shared memory environments. Existing state-of-the-art memory timing side channel mitigations have several key performance and security limitations. Prior schemes require onerous static bandwidth partitioning, extensive profiling phases, or simply fail to protect against attacks which exploit fine-grained timing and bank information. We present DAGguise, a defense mechanism which fully protects against memory timing side channels while allowing for dynamic traffic contention in order to achieve good performance. DAGguise utilizes a novel abstract memory access representation, the Directed Acyclic Request Graph (rDAG for short), to model memory access patterns which experience contention. DAGguise shapes a victim’s memory access patterns according to a publicly known rDAG obtained through a lightweight profiling stage, completely eliminating information leakage. We formally verify the security of DAGguise, proving that it maintains strong security guarantees. Moreover, by allowing dynamic traffic contention, DAGguise achieves a 12% overall system speedup relative to Fixed Service, which is the state-of-the-art mitigation mechanism, with up to a 20% relative speedup for co-located applications which do not require protection. We further claim that the principles of DAGguise can be generalized to protect against other types of scheduler-based timing side channels, such as those targeting on-chip networks, or functional units in SMT cores. @InProceedings{ASPLOS22p329, author = {Peter W. Deutsch and Yuheng Yang and Thomas Bourgeat and Jules Drean and Joel S. Emer and Mengjia Yan}, title = {DAGguise: Mitigating Memory Timing Side Channels}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {329--343}, doi = {10.1145/3503222.3507747}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Yang, Boyuan	ASPLOS '22: "Eavesdropping User Credentials ..." Eavesdropping User Credentials via GPU Side Channels on Smartphones Boyuan Yang, Ruirong Chen, Kai Huang, Jun Yang, and Wei Gao (University of Pittsburgh, USA) Graphics Processing Unit (GPU) on smartphones is an effective target for hardware attacks. In this paper, we present a new side channel attack on mobile GPUs of Android smartphones, allowing an unprivileged attacker to eavesdrop the user's credentials, such as login usernames and passwords, from their inputs through on-screen keyboard. Our attack targets on Qualcomm Adreno GPUs and investigate the amount of GPU overdraw when rendering the popups of user's key presses of inputs. Such GPU overdraw caused by each key press corresponds to unique variations of selected GPU performance counters, from which these key presses can be accurately inferred. Experiment results from practical use on multiple models of Android smartphones show that our attack can correctly infer more than 80% of user's credential inputs, but incur negligible amounts of computing overhead and network traffic on the victim device. To counter this attack, this paper suggests mitigations of access control on GPU performance counters, or applying obfuscations on the values of GPU performance counters. @InProceedings{ASPLOS22p285, author = {Boyuan Yang and Ruirong Chen and Kai Huang and Jun Yang and Wei Gao}, title = {Eavesdropping User Credentials via GPU Side Channels on Smartphones}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {285--299}, doi = {10.1145/3503222.3507757}, year = {2022}, } Publisher's Version Published Artifact Video Artifacts Available Artifacts Functional
	Yang, Jun	ASPLOS '22: "Eavesdropping User Credentials ..." Eavesdropping User Credentials via GPU Side Channels on Smartphones Boyuan Yang, Ruirong Chen, Kai Huang, Jun Yang, and Wei Gao (University of Pittsburgh, USA) Graphics Processing Unit (GPU) on smartphones is an effective target for hardware attacks. In this paper, we present a new side channel attack on mobile GPUs of Android smartphones, allowing an unprivileged attacker to eavesdrop the user's credentials, such as login usernames and passwords, from their inputs through on-screen keyboard. Our attack targets on Qualcomm Adreno GPUs and investigate the amount of GPU overdraw when rendering the popups of user's key presses of inputs. Such GPU overdraw caused by each key press corresponds to unique variations of selected GPU performance counters, from which these key presses can be accurately inferred. Experiment results from practical use on multiple models of Android smartphones show that our attack can correctly infer more than 80% of user's credential inputs, but incur negligible amounts of computing overhead and network traffic on the victim device. To counter this attack, this paper suggests mitigations of access control on GPU performance counters, or applying obfuscations on the values of GPU performance counters. @InProceedings{ASPLOS22p285, author = {Boyuan Yang and Ruirong Chen and Kai Huang and Jun Yang and Wei Gao}, title = {Eavesdropping User Credentials via GPU Side Channels on Smartphones}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {285--299}, doi = {10.1145/3503222.3507757}, year = {2022}, } Publisher's Version Published Artifact Video Artifacts Available Artifacts Functional ASPLOS '22: "AStitch: Enabling a New Multi-dimensional ..." AStitch: Enabling a New Multi-dimensional Optimization Space for Memory-Intensive ML Training and Inference on Modern SIMT Architectures Zhen Zheng, Xuanda Yang, Pengzhan Zhao, Guoping Long, Kai Zhu, Feiwen Zhu, Wenyi Zhao, Xiaoyong Liu, Jun Yang, Jidong Zhai, Shuaiwen Leon Song, and Wei Lin (Alibaba Group, China; Tsinghua University, China; University of Sydney, Australia) This work reveals that memory-intensive computation is a rising performance-critical factor in recent machine learning models. Due to a unique set of new challenges, existing ML optimizing compilers cannot perform efficient fusion under complex two-level dependencies combined with just-in-time demand. They face the dilemma of either performing costly fusion due to heavy redundant computation, or skipping fusion which results in massive number of kernels. Furthermore, they often suffer from low parallelism due to the lack of support for real-world production workloads with irregular tensor shapes. To address these rising challenges, we propose AStitch, a machine learning optimizing compiler that opens a new multi-dimensional optimization space for memory-intensive ML computations. It systematically abstracts four operator-stitching schemes while considering multi-dimensional optimization objectives, tackles complex computation graph dependencies with novel hierarchical data reuse, and efficiently processes various tensor shapes via adaptive thread mapping. Finally, AStitch provides just-in-time support incorporating our proposed optimizations for both ML training and inference. Although AStitch serves as a stand-alone compiler engine that is portable to any version of TensorFlow, its basic ideas can be generally applied to other ML frameworks and optimization compilers. Experimental results show that AStitch can achieve an average of 1.84x speedup (up to 2.73x) over the state-of-the-art Google's XLA solution across five production workloads. We also deploy AStitch onto a production cluster for ML workloads with thousands of GPUs. The system has been in operation for more than 10 months and saves about 20,000 GPU hours for 70,000 tasks per week. @InProceedings{ASPLOS22p359, author = {Zhen Zheng and Xuanda Yang and Pengzhan Zhao and Guoping Long and Kai Zhu and Feiwen Zhu and Wenyi Zhao and Xiaoyong Liu and Jun Yang and Jidong Zhai and Shuaiwen Leon Song and Wei Lin}, title = {AStitch: Enabling a New Multi-dimensional Optimization Space for Memory-Intensive ML Training and Inference on Modern SIMT Architectures}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {359--373}, doi = {10.1145/3503222.3507723}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Yang, Leon	ASPLOS '22: "TMO: Transparent Memory Offloading ..." TMO: Transparent Memory Offloading in Datacenters Johannes Weiner, Niket Agarwal, Dan Schatzberg, Leon Yang, Hao Wang, Blaise Sanouillet, Bikash Sharma, Tejun Heo, Mayank Jain, Chunqiang Tang, and Dimitrios Skarlatos (Meta, USA; Carnegie Mellon University, USA) The unrelenting growth of the memory needs of emerging datacenter applications, along with ever increasing cost and volatility of DRAM prices, has led to DRAM being a major infrastructure expense. Alternative technologies, such as NVMe SSDs and upcoming NVM devices, offer higher capacity than DRAM at a fraction of the cost and power. One promising approach is to transparently offload colder memory to cheaper memory technologies via kernel or hypervisor techniques. The key challenge, however, is to develop a datacenter-scale solution that is robust in dealing with diverse workloads and large performance variance of different offload devices such as compressed memory, SSD, and NVM. This paper presents TMO, Meta’s transparent memory offloading solution for heterogeneous datacenter environments. TMO introduces a new Linux kernel mechanism that directly measures in realtime the lost work due to resource shortage across CPU, memory, and I/O. Guided by this information and without any prior application knowledge, TMO automatically adjusts how much memory to offload to heterogeneous devices (e.g., compressed memory or SSD) according to the device’s performance characteristics and the application’s sensitivity to memory-access slowdown. TMO holistically identifies offloading opportunities from not only the application containers but also the sidecar containers that provide infrastructure-level functions. To maximize memory savings, TMO targets both anonymous memory and file cache, and balances the swap-in rate of anonymous memory and the reload rate of file pages that were recently evicted from the file cache. TMO has been running in production for more than a year, and has saved between 20-32% of the total memory across millions of servers in our large datacenter fleet. We have successfully upstreamed TMO into the Linux kernel. @InProceedings{ASPLOS22p609, author = {Johannes Weiner and Niket Agarwal and Dan Schatzberg and Leon Yang and Hao Wang and Blaise Sanouillet and Bikash Sharma and Tejun Heo and Mayank Jain and Chunqiang Tang and Dimitrios Skarlatos}, title = {TMO: Transparent Memory Offloading in Datacenters}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {609--621}, doi = {10.1145/3503222.3507731}, year = {2022}, } Publisher's Version
	Yang, Xuanda	ASPLOS '22: "AStitch: Enabling a New Multi-dimensional ..." AStitch: Enabling a New Multi-dimensional Optimization Space for Memory-Intensive ML Training and Inference on Modern SIMT Architectures Zhen Zheng, Xuanda Yang, Pengzhan Zhao, Guoping Long, Kai Zhu, Feiwen Zhu, Wenyi Zhao, Xiaoyong Liu, Jun Yang, Jidong Zhai, Shuaiwen Leon Song, and Wei Lin (Alibaba Group, China; Tsinghua University, China; University of Sydney, Australia) This work reveals that memory-intensive computation is a rising performance-critical factor in recent machine learning models. Due to a unique set of new challenges, existing ML optimizing compilers cannot perform efficient fusion under complex two-level dependencies combined with just-in-time demand. They face the dilemma of either performing costly fusion due to heavy redundant computation, or skipping fusion which results in massive number of kernels. Furthermore, they often suffer from low parallelism due to the lack of support for real-world production workloads with irregular tensor shapes. To address these rising challenges, we propose AStitch, a machine learning optimizing compiler that opens a new multi-dimensional optimization space for memory-intensive ML computations. It systematically abstracts four operator-stitching schemes while considering multi-dimensional optimization objectives, tackles complex computation graph dependencies with novel hierarchical data reuse, and efficiently processes various tensor shapes via adaptive thread mapping. Finally, AStitch provides just-in-time support incorporating our proposed optimizations for both ML training and inference. Although AStitch serves as a stand-alone compiler engine that is portable to any version of TensorFlow, its basic ideas can be generally applied to other ML frameworks and optimization compilers. Experimental results show that AStitch can achieve an average of 1.84x speedup (up to 2.73x) over the state-of-the-art Google's XLA solution across five production workloads. We also deploy AStitch onto a production cluster for ML workloads with thousands of GPUs. The system has been in operation for more than 10 months and saves about 20,000 GPU hours for 70,000 tasks per week. @InProceedings{ASPLOS22p359, author = {Zhen Zheng and Xuanda Yang and Pengzhan Zhao and Guoping Long and Kai Zhu and Feiwen Zhu and Wenyi Zhao and Xiaoyong Liu and Jun Yang and Jidong Zhai and Shuaiwen Leon Song and Wei Lin}, title = {AStitch: Enabling a New Multi-dimensional Optimization Space for Memory-Intensive ML Training and Inference on Modern SIMT Architectures}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {359--373}, doi = {10.1145/3503222.3507723}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Yang, Yanan	ASPLOS '22: "INFless: A Native Serverless ..." INFless: A Native Serverless System for Low-Latency, High-Throughput Inference Yanan Yang, Laiping Zhao, Yiming Li, Huanyu Zhang, Jie Li, Mingyang Zhao, Xingzhen Chen, and Keqiu Li (Tianjin University, China; 58.com, China) Modern websites increasingly rely on machine learning (ML) to improve their business efficiency. Developing and maintaining ML services incurs high costs for developers. Although serverless systems are a promising solution to reduce costs, we find that the current general purpose serverless systems cannot meet the low latency, high throughput demands of ML services. While simply ”patching” general serverless systems does not resolve the problem completely, we propose that such a system should natively combine the features of inference with a serverless paradigm. We present INFless, the first ML domain-specific serverless platform. It provides a unified, heterogeneous resource abstraction between CPU and accelerators, and achieves high throughput using built-in batching and non-uniform scaling mechanisms. It also supports low latency through coordinated management of batch queuing time, execution time and coldstart rate. We evaluate INFless using both a local cluster testbed and a large-scale simulation. Experimental results show that INFless outperforms state-of-the-art systems by 2×-5× on system throughput, meeting the latency goals of ML services. @InProceedings{ASPLOS22p768, author = {Yanan Yang and Laiping Zhao and Yiming Li and Huanyu Zhang and Jie Li and Mingyang Zhao and Xingzhen Chen and Keqiu Li}, title = {INFless: A Native Serverless System for Low-Latency, High-Throughput Inference}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {768--781}, doi = {10.1145/3503222.3507709}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available
	Yang, Yuheng	ASPLOS '22: "DAGguise: Mitigating Memory ..." DAGguise: Mitigating Memory Timing Side Channels Peter W. Deutsch, Yuheng Yang, Thomas Bourgeat, Jules Drean, Joel S. Emer, and Mengjia Yan (Massachusetts Institute of Technology, USA; NVIDIA, USA) This paper studies the mitigation of memory timing side channels, where attackers utilize contention within DRAM controllers to infer a victim’s secrets. Already practical, this class of channels poses an important challenge to secure computing in shared memory environments. Existing state-of-the-art memory timing side channel mitigations have several key performance and security limitations. Prior schemes require onerous static bandwidth partitioning, extensive profiling phases, or simply fail to protect against attacks which exploit fine-grained timing and bank information. We present DAGguise, a defense mechanism which fully protects against memory timing side channels while allowing for dynamic traffic contention in order to achieve good performance. DAGguise utilizes a novel abstract memory access representation, the Directed Acyclic Request Graph (rDAG for short), to model memory access patterns which experience contention. DAGguise shapes a victim’s memory access patterns according to a publicly known rDAG obtained through a lightweight profiling stage, completely eliminating information leakage. We formally verify the security of DAGguise, proving that it maintains strong security guarantees. Moreover, by allowing dynamic traffic contention, DAGguise achieves a 12% overall system speedup relative to Fixed Service, which is the state-of-the-art mitigation mechanism, with up to a 20% relative speedup for co-located applications which do not require protection. We further claim that the principles of DAGguise can be generalized to protect against other types of scheduler-based timing side channels, such as those targeting on-chip networks, or functional units in SMT cores. @InProceedings{ASPLOS22p329, author = {Peter W. Deutsch and Yuheng Yang and Thomas Bourgeat and Jules Drean and Joel S. Emer and Mengjia Yan}, title = {DAGguise: Mitigating Memory Timing Side Channels}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {329--343}, doi = {10.1145/3503222.3507747}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Yik, Jason	ASPLOS '22: "SparseCore: Stream ISA and ..." SparseCore: Stream ISA and Processor Specialization for Sparse Computation Gengyu Rao, Jingji Chen, Jason Yik, and Xuehai Qian (University of Southern California, USA) Computation on sparse data is becoming increasingly important for many applications. Recent sparse computation accelerators are designed for specific algorithm/application, making them inflexible with software optimizations. This paper proposes SparseCore, the first general-purpose processor extension for sparse computation that can flexibly accelerate complex code patterns and fast-evolving algorithms. We extend the instruction set architecture (ISA) to make stream or sparse vector first-class citizens, and develop efficient architectural components to support the stream ISA. The novel ISA extension intrinsically operates on streams, realizing both efficient data movement and computation. The simulation results show that SparseCore achieves significant speedups for sparse tensor computation and graph pattern computation. @InProceedings{ASPLOS22p186, author = {Gengyu Rao and Jingji Chen and Jason Yik and Xuehai Qian}, title = {SparseCore: Stream ISA and Processor Specialization for Sparse Computation}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {186--199}, doi = {10.1145/3503222.3507705}, year = {2022}, } Publisher's Version
	Younis, Ed	ASPLOS '22: "QUEST: Systematically Approximating ..." QUEST: Systematically Approximating Quantum Circuits for Higher Output Fidelity Tirthak Patel, Ed Younis, Costin Iancu, Wibe de Jong, and Devesh Tiwari (Northeastern University, USA; Lawrence Berkeley National Laboratory, USA) We present QUEST, a procedure to systematically generate approximations for quantum circuits to reduce their CNOT gate count. Our approach employs circuit partitioning for scalability with procedures to 1) reduce circuit length using approximate synthesis, 2) improve fidelity by running circuits that represent key samples in the approximation space, and 3) reason about approximation upper bound. Our evaluation results indicate that our approach of "dissimilar" approximations provides close fidelity to the original circuit. Overall, the results indicate that QUEST can reduce CNOT gate count by 30-80% on ideal systems and decrease the impact of noise on existing and near-future quantum systems. @InProceedings{ASPLOS22p514, author = {Tirthak Patel and Ed Younis and Costin Iancu and Wibe de Jong and Devesh Tiwari}, title = {QUEST: Systematically Approximating Quantum Circuits for Higher Output Fidelity}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {514--528}, doi = {10.1145/3503222.3507739}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional
	Yuan, Yongjie	ASPLOS '22: "CoolEdge: Hotspot-Relievable ..." CoolEdge: Hotspot-Relievable Warm Water Cooling for Energy-Efficient Edge Datacenters Qiangyu Pei, Shutong Chen, Qixia Zhang, Xinhui Zhu, Fangming Liu, Ziyang Jia, Yishuo Wang, and Yongjie Yuan (Huazhong University of Science and Technology, China) As the computing frontier drifts to the edge, edge datacenters play a crucial role in supporting various real-time applications. Different from cloud datacenters, the requirements of proximity to end-users, high density, and heterogeneity, present new challenges to cool the edge datacenters efficiently. Although warm water cooling has become a promising cooling technique for this infrastructure, the one-size-fits-all cooling control would lower the cooling efficiency considerably because of the severe thermal imbalance across servers, hardware, and even inside one hardware component in an edge datacenter. In this work, we propose CoolEdge, a hotspot-relievable warm water cooling system for improving the cooling efficiency and saving costs of edge datacenters. Specifically, through the elaborate design of water circulations, CoolEdge can dynamically adjust the water temperature and flow rate for each heterogeneous hardware component to eliminate the hardware-level hotspots. By redesigning cold plates, CoolEdge can quickly disperse the chip-level hotspots without manual intervention. We further quantify the power saving achieved by the warm water cooling theoretically, and propose a custom-designed cooling solution to decide an appropriate water temperature and flow rate periodically. Based on a hardware prototype and real-world traces from SURFsara, the evaluation results show that CoolEdge reduces the cooling energy by 81.81% and 71.92%, respectively, compared with conventional and state-of-the-art water cooling systems. @InProceedings{ASPLOS22p814, author = {Qiangyu Pei and Shutong Chen and Qixia Zhang and Xinhui Zhu and Fangming Liu and Ziyang Jia and Yishuo Wang and Yongjie Yuan}, title = {CoolEdge: Hotspot-Relievable Warm Water Cooling for Energy-Efficient Edge Datacenters}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {814--829}, doi = {10.1145/3503222.3507713}, year = {2022}, } Publisher's Version
	Zang, Binyu	ASPLOS '22: "Serverless Computing on Heterogeneous ..." Serverless Computing on Heterogeneous Computers Dong Du, Qingyuan Liu, Xueqiang Jiang, Yubin Xia, Binyu Zang, and Haibo Chen (Shanghai Jiao Tong University, China; Shanghai Artificial Intelligence Laboratory, China) Existing serverless computing platforms are built upon homogeneous computers, limiting the function density and restricting serverless computing to limited scenarios. We introduce Molecule, the first serverless computing system utilizing heterogeneous computers. Molecule enables both general-purpose devices (e.g., Nvidia DPU) and domain-specific accelerators (e.g., FPGA and GPU) for serverless applications that significantly improve function density (50% higher) and application performance (up to 34.6x). To achieve these results, we first propose XPU-Shim, a distributed shim to bridge the gap between underlying multi-OS systems (when using general-purpose devices) and our serverless runtime (i.e., Molecule). We further introduce vectorized sandbox, a sandbox abstraction to abstract hardware heterogeneity (when using domain-specific accelerators). Moreover, we also review state-of-the-art serverless optimizations on startup and communication latency and overcome the challenges to implement them on heterogeneous computers. We have implemented Molecule on real platforms with Nvidia DPUs and Xilinx FPGAs and evaluate it using benchmarks and real-world applications. @InProceedings{ASPLOS22p797, author = {Dong Du and Qingyuan Liu and Xueqiang Jiang and Yubin Xia and Binyu Zang and Haibo Chen}, title = {Serverless Computing on Heterogeneous Computers}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {797--813}, doi = {10.1145/3503222.3507732}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Zhai, Jidong	ASPLOS '22: "AStitch: Enabling a New Multi-dimensional ..." AStitch: Enabling a New Multi-dimensional Optimization Space for Memory-Intensive ML Training and Inference on Modern SIMT Architectures Zhen Zheng, Xuanda Yang, Pengzhan Zhao, Guoping Long, Kai Zhu, Feiwen Zhu, Wenyi Zhao, Xiaoyong Liu, Jun Yang, Jidong Zhai, Shuaiwen Leon Song, and Wei Lin (Alibaba Group, China; Tsinghua University, China; University of Sydney, Australia) This work reveals that memory-intensive computation is a rising performance-critical factor in recent machine learning models. Due to a unique set of new challenges, existing ML optimizing compilers cannot perform efficient fusion under complex two-level dependencies combined with just-in-time demand. They face the dilemma of either performing costly fusion due to heavy redundant computation, or skipping fusion which results in massive number of kernels. Furthermore, they often suffer from low parallelism due to the lack of support for real-world production workloads with irregular tensor shapes. To address these rising challenges, we propose AStitch, a machine learning optimizing compiler that opens a new multi-dimensional optimization space for memory-intensive ML computations. It systematically abstracts four operator-stitching schemes while considering multi-dimensional optimization objectives, tackles complex computation graph dependencies with novel hierarchical data reuse, and efficiently processes various tensor shapes via adaptive thread mapping. Finally, AStitch provides just-in-time support incorporating our proposed optimizations for both ML training and inference. Although AStitch serves as a stand-alone compiler engine that is portable to any version of TensorFlow, its basic ideas can be generally applied to other ML frameworks and optimization compilers. Experimental results show that AStitch can achieve an average of 1.84x speedup (up to 2.73x) over the state-of-the-art Google's XLA solution across five production workloads. We also deploy AStitch onto a production cluster for ML workloads with thousands of GPUs. The system has been in operation for more than 10 months and saves about 20,000 GPU hours for 70,000 tasks per week. @InProceedings{ASPLOS22p359, author = {Zhen Zheng and Xuanda Yang and Pengzhan Zhao and Guoping Long and Kai Zhu and Feiwen Zhu and Wenyi Zhao and Xiaoyong Liu and Jun Yang and Jidong Zhai and Shuaiwen Leon Song and Wei Lin}, title = {AStitch: Enabling a New Multi-dimensional Optimization Space for Memory-Intensive ML Training and Inference on Modern SIMT Architectures}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {359--373}, doi = {10.1145/3503222.3507723}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced ASPLOS '22: "Suppressing ZZ Crosstalk of ..." Suppressing ZZ Crosstalk of Quantum Computers through Pulse and Scheduling Co-Optimization Lei Xie, Jidong Zhai, ZhenXing Zhang, Jonathan Allcock, Shengyu Zhang, and Yi-Cong Zheng (Tsinghua University, China; Tencent Quantum Laboratory, China) Noise is a significant obstacle to quantum computing, and ZZ crosstalk is one of the most destructive types of noise affecting superconducting qubits. Previous approaches to suppressing ZZ crosstalk have mainly relied on specific chip design that can complicate chip fabrication and aggravate decoherence. To some extent, special chip design can be avoided by relying on pulse optimization to suppress ZZ crosstalk. However, existing approaches are non-scalable, as their required time and memory grow exponentially with the number of qubits involved. To address the above problems, we propose a scalable approach by co-optimizing pulses and scheduling. We optimize pulses to offer an ability to suppress ZZ crosstalk surrounding a gate, and then design scheduling strategies to exploit this ability and achieve suppression across the whole circuit. A main advantage of such co-optimization is that it does not require special hardware support. Besides, we implement our approach as a general framework that is compatible with different pulse optimization methods. We have conducted extensive evaluations by simulation and on a real quantum computer. Simulation results show that our proposal can improve the fidelity of quantum computing on 4∼12 qubits by up to 81× (11× on average). Ramsey experiments on a real quantum computer also demonstrate that our method can eliminate the effect of ZZ crosstalk to a great extent. @InProceedings{ASPLOS22p499, author = {Lei Xie and Jidong Zhai and ZhenXing Zhang and Jonathan Allcock and Shengyu Zhang and Yi-Cong Zheng}, title = {Suppressing ZZ Crosstalk of Quantum Computers through Pulse and Scheduling Co-Optimization}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {499--513}, doi = {10.1145/3503222.3507761}, year = {2022}, } Publisher's Version
	Zhang, Dan	ASPLOS '22: "A Full-Stack Search Technique ..." A Full-Stack Search Technique for Domain Optimized Deep Learning Accelerators Dan Zhang, Safeen Huda, Ebrahim Songhori, Kartik Prabhu, Quoc Le, Anna Goldie, and Azalia Mirhoseini (Google Brain, USA; Google, USA; Stanford University, USA) The rapidly-changing deep learning landscape presents a unique opportunity for building inference accelerators optimized for specific datacenter-scale workloads. We propose Full-stack Accelerator Search Technique (FAST), a hardware accelerator search framework that defines a broad optimization environment covering key design decisions within the hardware-software stack, including hardware datapath, software scheduling, and compiler passes such as operation fusion and tensor padding. In this paper, we analyze bottlenecks in state-of-the-art vision and natural language processing (NLP) models, including EfficientNet and BERT, and use FAST to design accelerators capable of addressing these bottlenecks. FAST-generated accelerators optimized for single workloads improve Perf/TDP by 3.7× on average across all benchmarks compared to TPU-v3. A FAST-generated accelerator optimized for serving a suite of workloads improves Perf/TDP by 2.4× on average compared to TPU-v3. Our return on investment analysis shows that FAST-generated accelerators can potentially be practical for moderate-sized datacenter deployments. @InProceedings{ASPLOS22p27, author = {Dan Zhang and Safeen Huda and Ebrahim Songhori and Kartik Prabhu and Quoc Le and Anna Goldie and Azalia Mirhoseini}, title = {A Full-Stack Search Technique for Domain Optimized Deep Learning Accelerators}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {27--42}, doi = {10.1145/3503222.3507767}, year = {2022}, } Publisher's Version
	Zhang, Haotian	ASPLOS '22: "One Size Does Not Fit All: ..." One Size Does Not Fit All: Security Hardening of MIPS Embedded Systems via Static Binary Debloating for Shared Libraries Haotian Zhang, Mengfei Ren, Yu Lei, and Jiang Ming (University of Texas at Arlington, USA) Embedded systems have become prominent targets for cyberattacks. To exploit firmware’s memory corruption vulnerabilities, cybercriminals harvest reusable code gadgets from the large shared library codebase (e.g., uClibc). Unfortunately, unlike their desktop counterparts, embedded systems lack essential computing resources to enforce security hardening techniques. Recently, we have witnessed a surge of software debloating as a new defense mechanism against code-reuse attacks; it erases unused code to significantly diminish the possibilities of constructing reusable gadgets. Because of the single firmware image update style, static library debloating shows promise to fortify embedded systems without compromising performance and forward compatibility. However, static library debloating on stripped binaries (e.g., firmware’s shared libraries) is still an enormous challenge. In this paper, we show that this challenge is not insurmountable for MIPS firmware. We develop a novel system, named uTrimmer, to identify and wipe out unused basic blocks from shared libraries’ binary code, without causing additional runtime overhead or memory consumption. We propose a new method to identify address-taken blocks/functions, which further help us maintain an inter-procedural control flow graph to conservatively include library code that could be potentially used by firmware. By capturing address access patterns for position-independent code, we circumvent the challenge of determining code-pointer targets and safely eliminate unused code. We run uTrimmer to debloat shared libraries for SPEC CPU2017 benchmarks, popular firmware applications (e.g., Apache, BusyBox, and OpenSSL), and a real-world wireless router firmware image. Our experiments show that not only does uTrimmer deliver functional programs, but also it can cut the exposed code surface and eliminate various reusable code gadgets remarkably. uTrimmer’s debloating capability can compete with the static linking results. @InProceedings{ASPLOS22p255, author = {Haotian Zhang and Mengfei Ren and Yu Lei and Jiang Ming}, title = {One Size Does Not Fit All: Security Hardening of MIPS Embedded Systems via Static Binary Debloating for Shared Libraries}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {255--270}, doi = {10.1145/3503222.3507768}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Zhang, Haoyang	ASPLOS '22: "Debugging in the Brave New ..." Debugging in the Brave New World of Reconfigurable Hardware Jiacheng Ma, Gefei Zuo, Kevin Loughlin, Haoyang Zhang, Andrew Quinn, and Baris Kasikci (University of Michigan, USA; University of California at Santa Cruz, USA) Software and hardware development cycles have traditionally been quite distinct. Software allows post-deployment patches, which leads to a rapid development cycle. In contrast, hardware bugs that are found after fabrication are extremely costly to fix (and sometimes even unfixable), so the traditional hardware development cycle involves massive investment in extensive simulation and formal verification. Reconfigurable hardware, such as a Field Programmable Gate Array (FPGA), promises to propel hardware development towards an agile software-like development approach, since it enables a hardware developer to patch bugs that are detected during on-chip testing or in production. Unfortunately, FPGA programmers lack bug localization tools amenable to this rapid development cycle, since past tools mainly find bugs via simulation and verification. To develop hardware bug localization tools for a rapid development cycle, a thorough understanding of the symptoms, root causes, and fixes of hardware bugs is needed. In this paper, we first study bugs in existing FPGA designs and produce a testbed of reliably-reproducible bugs. We classify the bugs according to their intrinsic properties, symptoms, and root causes. We demonstrate that many hardware bugs are comparable to software bug counterparts, and would benefit from similar techniques for bug diagnosis and repair. Based upon our findings, we build a novel collection of hybrid static/dynamic program analysis and monitoring tools for debugging FPGA designs, showing that our tools enable a software-like development cycle by effectively reducing developers' manual efforts for bug localization. @InProceedings{ASPLOS22p946, author = {Jiacheng Ma and Gefei Zuo and Kevin Loughlin and Haoyang Zhang and Andrew Quinn and Baris Kasikci}, title = {Debugging in the Brave New World of Reconfigurable Hardware}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {946--962}, doi = {10.1145/3503222.3507701}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Zhang, Huanyu	ASPLOS '22: "INFless: A Native Serverless ..." INFless: A Native Serverless System for Low-Latency, High-Throughput Inference Yanan Yang, Laiping Zhao, Yiming Li, Huanyu Zhang, Jie Li, Mingyang Zhao, Xingzhen Chen, and Keqiu Li (Tianjin University, China; 58.com, China) Modern websites increasingly rely on machine learning (ML) to improve their business efficiency. Developing and maintaining ML services incurs high costs for developers. Although serverless systems are a promising solution to reduce costs, we find that the current general purpose serverless systems cannot meet the low latency, high throughput demands of ML services. While simply ”patching” general serverless systems does not resolve the problem completely, we propose that such a system should natively combine the features of inference with a serverless paradigm. We present INFless, the first ML domain-specific serverless platform. It provides a unified, heterogeneous resource abstraction between CPU and accelerators, and achieves high throughput using built-in batching and non-uniform scaling mechanisms. It also supports low latency through coordinated management of batch queuing time, execution time and coldstart rate. We evaluate INFless using both a local cluster testbed and a large-scale simulation. Experimental results show that INFless outperforms state-of-the-art systems by 2×-5× on system throughput, meeting the latency goals of ML services. @InProceedings{ASPLOS22p768, author = {Yanan Yang and Laiping Zhao and Yiming Li and Huanyu Zhang and Jie Li and Mingyang Zhao and Xingzhen Chen and Keqiu Li}, title = {INFless: A Native Serverless System for Low-Latency, High-Throughput Inference}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {768--781}, doi = {10.1145/3503222.3507709}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available
	Zhang, Jialiang	ASPLOS '22: "Software-Defined Address Mapping: ..." Software-Defined Address Mapping: A Case on 3D Memory Jialiang Zhang, Michael Swift, and Jing (Jane) Li (University of Pennsylvania, USA; University of Wisconsin-Madison, USA) 3D-stacking memory such as High-Bandwidth Memory (HBM) and Hybrid Memory Cube (HMC) provides orders of magnitude more bandwidth and significantly increased channel-level parallelism (CLP) due to its new parallel memory architecture. However, it is challenging to fully exploit the abundant CLP for performance as the bandwidth utilization is highly dependent on address mapping in the memory controller. Unfortunately, CLP is very sensitive to a program’s data access pattern, which is not made available to OS/hardware by existing mechanisms. In this work, we address these challenges with software-defined address mapping (SDAM) that, for the first time, enables user program to obtain a direct control of the low-level memory hardware in a more intelligent and fine-grained manner. In particular, we develop new mechanisms that can effectively communicate a program’s data access properties to the OS and hardware and to use it to control data placement in hardware. To guarantee correctness and reduce overhead in storage and performance, we extend Linux kernel and C-language memory allocators to support multiple address mappings. For advanced system optimization, we develop machine learning methods that can automatically identify access patterns of major variables in a program and cluster these with similar access patterns to reduce the overhead for SDAM. We demonstrate the benefits of our design on real system prototype, comprising (1) a RISC-V processor, near memory accelerators and HBM modules using Xilinx FPGA platform, and (2) modified Linux and glibc. Our evaluation on standard CPU benchmarks and data-intensive benchmarks (for both CPU and accelerators) demonstrates 1.41×, 1.84× speedup on CPU and 2.58× on near memory accelerators in our system with SDAM compared to a baseline system that uses a fixed address mapping. @InProceedings{ASPLOS22p70, author = {Jialiang Zhang and Michael Swift and Jing (Jane) Li}, title = {Software-Defined Address Mapping: A Case on 3D Memory}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {70--83}, doi = {10.1145/3503222.3507774}, year = {2022}, } Publisher's Version
	Zhang, Nicholas	ASPLOS '22: "NASPipe: High Performance ..." NASPipe: High Performance and Reproducible Pipeline Parallel Supernet Training via Causal Synchronous Parallelism Shixiong Zhao, Fanxin Li, Xusheng Chen, Tianxiang Shen, Li Chen, Sen Wang, Nicholas Zhang, Cheng Li, and Heming Cui (University of Hong Kong, China; Huawei Technologies, China; University of Science and Technology of China, China) Supernet training, a prevalent and important paradigm in Neural Architecture Search, embeds the whole DNN architecture search space into one monolithic supernet, iteratively activates a subset of the supernet (i.e., a subnet) for fitting each batch of data, and searches a high-quality subnet which meets specific requirements. Although training subnets in parallel on multiple GPUs is desirable for acceleration, there inherently exists a race hazard that concurrent subnets may access the same DNN layers. Existing systems support neither efficiently parallelizing subnets’ training executions, nor resolving the race hazard deterministically, leading to unreproducible training procedures and potentiallly non-trivial accuracy loss. We present NASPipe, the first high-performance and reproducible distributed supernet training system via causal synchronous parallel (CSP) pipeline scheduling abstraction: NASPipe partitions a supernet across GPUs and concurrently executes multiple generated sub-tasks (subnets) in a pipelined manner; meanwhile, it oversees the correlations between the subnets and deterministically resolves any causal dependency caused by subnets’ layer sharing. To obtain high performance, NASPipe’s CSP scheduler exploits the fact that the larger a supernet spans, the fewer dependencies manifest between chronologically close subnets; therefore, it aggressively schedules the subnets with larger chronological orders into execution, only if they are not causally dependent on unfinished precedent subnets. Moreover, to relieve the excessive GPU memory burden for holding the whole supernet’s parameters, NASPipe uses a context switch technique that stashes the whole supernet in CPU memory, precisely predicts the subnets’ schedule, and pre-fetches/evicts a subnet before/after its execution. The evaluation shows that NASPipe is the only system that retains supernet training reproducibility, while achieving a comparable and even higher performance (up to 7.8X) compared to three recent pipeline training systems (e.g., GPipe). @InProceedings{ASPLOS22p374, author = {Shixiong Zhao and Fanxin Li and Xusheng Chen and Tianxiang Shen and Li Chen and Sen Wang and Nicholas Zhang and Cheng Li and Heming Cui}, title = {NASPipe: High Performance and Reproducible Pipeline Parallel Supernet Training via Causal Synchronous Parallelism}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {374--387}, doi = {10.1145/3503222.3507735}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Zhang, Qian	ASPLOS '22: "HeteroGen: Transpiling C to ..." HeteroGen: Transpiling C to Heterogeneous HLS Code with Automated Test Generation and Program Repair Qian Zhang, Jiyuan Wang, Guoqing Harry Xu, and Miryung Kim (University of California at Los Angeles, USA) Despite the trend of incorporating heterogeneity and specialization in hardware, the development of heterogeneous applications is limited to a handful of engineers with deep hardware expertise. We propose HeteroGen that takes C/C++ code as input and automatically generates an HLS version with test behavior preservation and better performance. Key to the success of HeteroGen is adapting the idea of search-based program repair to the heterogeneous computing domain, while addressing two technical challenges. First, the turn-around time of HLS compilation and simulation is much longer than the usual C/C++ compilation and execution time; therefore, HeteroGen applies pattern-oriented program edits guided by common fix patterns and their dependences. Second, behavior and performance checking requires testing, but test cases are often unavailable. Thus, HeteroGen auto-generates test inputs suitable for checking C to HLS-C conversion errors, while providing high branch coverage for the original C code. An evaluation of HeteroGen shows that it produces an HLS-compatible version for nine out of ten real-world heterogeneous applications fully automatically, applying up to 438 lines of edits to produce an HLS version 1.63x faster than the original version. @InProceedings{ASPLOS22p1017, author = {Qian Zhang and Jiyuan Wang and Guoqing Harry Xu and Miryung Kim}, title = {HeteroGen: Transpiling C to Heterogeneous HLS Code with Automated Test Generation and Program Repair}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1017--1029}, doi = {10.1145/3503222.3507748}, year = {2022}, } Publisher's Version Published Artifact Info Artifacts Available Artifacts Functional Results Reproduced
	Zhang, Qixia	ASPLOS '22: "CoolEdge: Hotspot-Relievable ..." CoolEdge: Hotspot-Relievable Warm Water Cooling for Energy-Efficient Edge Datacenters Qiangyu Pei, Shutong Chen, Qixia Zhang, Xinhui Zhu, Fangming Liu, Ziyang Jia, Yishuo Wang, and Yongjie Yuan (Huazhong University of Science and Technology, China) As the computing frontier drifts to the edge, edge datacenters play a crucial role in supporting various real-time applications. Different from cloud datacenters, the requirements of proximity to end-users, high density, and heterogeneity, present new challenges to cool the edge datacenters efficiently. Although warm water cooling has become a promising cooling technique for this infrastructure, the one-size-fits-all cooling control would lower the cooling efficiency considerably because of the severe thermal imbalance across servers, hardware, and even inside one hardware component in an edge datacenter. In this work, we propose CoolEdge, a hotspot-relievable warm water cooling system for improving the cooling efficiency and saving costs of edge datacenters. Specifically, through the elaborate design of water circulations, CoolEdge can dynamically adjust the water temperature and flow rate for each heterogeneous hardware component to eliminate the hardware-level hotspots. By redesigning cold plates, CoolEdge can quickly disperse the chip-level hotspots without manual intervention. We further quantify the power saving achieved by the warm water cooling theoretically, and propose a custom-designed cooling solution to decide an appropriate water temperature and flow rate periodically. Based on a hardware prototype and real-world traces from SURFsara, the evaluation results show that CoolEdge reduces the cooling energy by 81.81% and 71.92%, respectively, compared with conventional and state-of-the-art water cooling systems. @InProceedings{ASPLOS22p814, author = {Qiangyu Pei and Shutong Chen and Qixia Zhang and Xinhui Zhu and Fangming Liu and Ziyang Jia and Yishuo Wang and Yongjie Yuan}, title = {CoolEdge: Hotspot-Relievable Warm Water Cooling for Energy-Efficient Edge Datacenters}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {814--829}, doi = {10.1145/3503222.3507713}, year = {2022}, } Publisher's Version
	Zhang, Shengyu	ASPLOS '22: "Suppressing ZZ Crosstalk of ..." Suppressing ZZ Crosstalk of Quantum Computers through Pulse and Scheduling Co-Optimization Lei Xie, Jidong Zhai, ZhenXing Zhang, Jonathan Allcock, Shengyu Zhang, and Yi-Cong Zheng (Tsinghua University, China; Tencent Quantum Laboratory, China) Noise is a significant obstacle to quantum computing, and ZZ crosstalk is one of the most destructive types of noise affecting superconducting qubits. Previous approaches to suppressing ZZ crosstalk have mainly relied on specific chip design that can complicate chip fabrication and aggravate decoherence. To some extent, special chip design can be avoided by relying on pulse optimization to suppress ZZ crosstalk. However, existing approaches are non-scalable, as their required time and memory grow exponentially with the number of qubits involved. To address the above problems, we propose a scalable approach by co-optimizing pulses and scheduling. We optimize pulses to offer an ability to suppress ZZ crosstalk surrounding a gate, and then design scheduling strategies to exploit this ability and achieve suppression across the whole circuit. A main advantage of such co-optimization is that it does not require special hardware support. Besides, we implement our approach as a general framework that is compatible with different pulse optimization methods. We have conducted extensive evaluations by simulation and on a real quantum computer. Simulation results show that our proposal can improve the fidelity of quantum computing on 4∼12 qubits by up to 81× (11× on average). Ramsey experiments on a real quantum computer also demonstrate that our method can eliminate the effect of ZZ crosstalk to a great extent. @InProceedings{ASPLOS22p499, author = {Lei Xie and Jidong Zhai and ZhenXing Zhang and Jonathan Allcock and Shengyu Zhang and Yi-Cong Zheng}, title = {Suppressing ZZ Crosstalk of Quantum Computers through Pulse and Scheduling Co-Optimization}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {499--513}, doi = {10.1145/3503222.3507761}, year = {2022}, } Publisher's Version
	Zhang, Wei	ASPLOS '22: "Astraea: Towards QoS-Aware ..." Astraea: Towards QoS-Aware and Resource-Efficient Multi-stage GPU Services Wei Zhang, Quan Chen, Kaihua Fu, Ningxin Zheng, Zhiyi Huang, Jingwen Leng, and Minyi Guo (Shanghai Jiao Tong University, China; Microsoft Research, China; University of Otago, New Zealand) Multi-stage user-facing applications on GPUs are widely-used nowa- days, and are often implemented to be microservices. Prior re- search works are not applicable to ensuring QoS of GPU-based microservices due to the different communication patterns and shared resource contentions. We propose Astraea to manage GPU microservices considering the above factors. In Astraea, a microser- vice deployment policy is used to maximize the supported peak service load while ensuring the required QoS. To adaptively switch the communication methods between microservices according to different deployments, we propose an auto-scaling GPU communi- cation framework. The framework automatically scales based on the currently used hardware topology and microservice location, and adopts global memory-based techniques to reduce intra-GPU communication. Astraea increases the supported peak load by up to 82.3% while achieving the desired 99%-ile latency target compared with state-of-the-art solutions. @InProceedings{ASPLOS22p570, author = {Wei Zhang and Quan Chen and Kaihua Fu and Ningxin Zheng and Zhiyi Huang and Jingwen Leng and Minyi Guo}, title = {Astraea: Towards QoS-Aware and Resource-Efficient Multi-stage GPU Services}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {570--582}, doi = {10.1145/3503222.3507721}, year = {2022}, } Publisher's Version
	Zhang, Xuechen	ASPLOS '22: "NVAlloc: Rethinking Heap Metadata ..." NVAlloc: Rethinking Heap Metadata Management in Persistent Memory Allocators Zheng Dang, Shuibing He, Peiyi Hong, Zhenxin Li, Xuechen Zhang, Xian-He Sun, and Gang Chen (Zhejiang University, China; Washington State University, USA; Illinois Institute of Technology, USA) Persistent memory allocation is a fundamental building block for developing high-performance and in-memory applications. Existing persistent memory allocators suffer from suboptimal heap organizations that introduce repeated cache line flushes and small random accesses in persistent memory. Worse, many allocators use static slab segregation resulting in a dramatic increase in memory consumption when allocation request size is changed. In this paper, we design a novel allocator, named NVAlloc, to solve the above issues simultaneously. First, NVAlloc eliminates cache line reflushes by mapping contiguous data blocks in slabs to interleaved metadata entries stored in different cache lines. Second, it writes small metadata units to a persistent bookkeeping log in a sequential pattern to remove random heap metadata accesses in persistent memory. Third, instead of using static slab segregation, it supports slab morphing, which allows slabs to be transformed between size classes to significantly improve slab usage. NVAlloc is complementary to the existing consistency models. Results on 6 benchmarks demonstrate that NVAlloc improves the performance of state-of-the-art persistent memory allocators by up to 6.4x and 57x for small and large allocations, respectively. Using NVAlloc reduces memory usage by up to 57.8%. Besides, we integrate NVAlloc in a persistent FPTree. Compared to the state-of-the-art allocators, NVAlloc improves the performance of this application by up to 3.1x. @InProceedings{ASPLOS22p115, author = {Zheng Dang and Shuibing He and Peiyi Hong and Zhenxin Li and Xuechen Zhang and Xian-He Sun and Gang Chen}, title = {NVAlloc: Rethinking Heap Metadata Management in Persistent Memory Allocators}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {115--127}, doi = {10.1145/3503222.3507743}, year = {2022}, } Publisher's Version
	Zhang, Yiying	ASPLOS '22: "Clio: A Hardware-Software ..." Clio: A Hardware-Software Co-Designed Disaggregated Memory System Zhiyuan Guo, Yizhou Shan, Xuhao Luo, Yutong Huang, and Yiying Zhang (University of California at San Diego, USA) Memory disaggregation has attracted great attention recently because of its benefits in efficient memory utilization and ease of management. So far, memory disaggregation research has all taken one of two approaches: building/emulating memory nodes using regular servers or building them using raw memory devices with no processing power. The former incurs higher monetary cost and faces tail latency and scalability limitations, while the latter introduces performance, security, and management problems. Server-based memory nodes and memory nodes with no processing power are two extreme approaches. We seek a sweet spot in the middle by proposing a hardware-based memory disaggregation solution that has the right amount of processing power at memory nodes. Furthermore, we take a clean-slate approach by starting from the requirements of memory disaggregation and designing a memory-disaggregation-native system. We built Clio, a disaggregated memory system that virtualizes, protects, and manages disaggregated memory at hardware-based memory nodes. The Clio hardware includes a new virtual memory system, a customized network system, and a framework for computation offloading. In building Clio, we not only co-design OS functionalities, hardware architecture, and the network system, but also co-design compute nodes and memory nodes. Our FPGA prototype of Clio demonstrates that each memory node can achieve 100 Gbps throughput and an end-to-end latency of 2.5 µ s at median and 3.2 µ s at the 99th percentile. Clio also scales much better and has orders of magnitude lower tail latency than RDMA. It has 1.1× to 3.4× energy saving compared to CPU-based and SmartNIC-based disaggregated memory systems and is 2.7× faster than software-based SmartNIC solutions. @InProceedings{ASPLOS22p417, author = {Zhiyuan Guo and Yizhou Shan and Xuhao Luo and Yutong Huang and Yiying Zhang}, title = {Clio: A Hardware-Software Co-Designed Disaggregated Memory System}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {417--433}, doi = {10.1145/3503222.3507762}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional
	Zhang, Yongle	ASPLOS '22: "Efficiently Detecting Concurrency ..." Efficiently Detecting Concurrency Bugs in Persistent Memory Programs Zhangyu Chen, Yu Hua, Yongle Zhang, and Luochangqi Ding (Huazhong University of Science and Technology, China; Purdue University, USA) Due to the salient DRAM-comparable performance, TB-scale capacity, and non-volatility, persistent memory (PM) provides new opportunities for large-scale in-memory computing with instant crash recovery. However, programming PM systems is error-prone due to the existence of crash-consistency bugs, which are challenging to diagnose especially with concurrent programming widely adopted in PM applications to exploit hardware parallelism. Existing bug detection tools for DRAM-based concurrency issues cannot detect PM crash-consistency bugs because they are oblivious to PM operations and PM consistency. On the other hand, existing PM-specific debugging tools only focus on sequential PM programs and cannot effectively detect crash-consistency issues hidden in concurrent executions. In order to effectively detect crash-consistency bugs that only manifest in concurrent executions, we propose PMRace, the first PM-specific concurrency bug detection tool. We identify and define two new types of concurrent crash-consistency bugs: PM Inter-thread Inconsistency and PM Synchronization Inconsistency. In particular, PMRace adopts PM-aware and coverage-guided fuzz testing to explore PM program executions. For PM Inter-thread Inconsistency, which denotes the data inconsistency hidden in thread interleavings, PMRace performs PM-aware interleaving exploration and thread scheduling to drive the execution towards executions that reveal such inconsistencies. For PM Synchronization Inconsistency between persisted synchronization variables and program data, PMRace identifies the inconsistency during interleaving exploration. The post-failure validation reduces the false positives that come from custom crash recovery mechanisms. PMRace has found 14 bugs (10 new bugs) in real-world concurrent PM systems including PM-version memcached. @InProceedings{ASPLOS22p873, author = {Zhangyu Chen and Yu Hua and Yongle Zhang and Luochangqi Ding}, title = {Efficiently Detecting Concurrency Bugs in Persistent Memory Programs}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {873--887}, doi = {10.1145/3503222.3507755}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Zhang, ZhenXing	ASPLOS '22: "Suppressing ZZ Crosstalk of ..." Suppressing ZZ Crosstalk of Quantum Computers through Pulse and Scheduling Co-Optimization Lei Xie, Jidong Zhai, ZhenXing Zhang, Jonathan Allcock, Shengyu Zhang, and Yi-Cong Zheng (Tsinghua University, China; Tencent Quantum Laboratory, China) Noise is a significant obstacle to quantum computing, and ZZ crosstalk is one of the most destructive types of noise affecting superconducting qubits. Previous approaches to suppressing ZZ crosstalk have mainly relied on specific chip design that can complicate chip fabrication and aggravate decoherence. To some extent, special chip design can be avoided by relying on pulse optimization to suppress ZZ crosstalk. However, existing approaches are non-scalable, as their required time and memory grow exponentially with the number of qubits involved. To address the above problems, we propose a scalable approach by co-optimizing pulses and scheduling. We optimize pulses to offer an ability to suppress ZZ crosstalk surrounding a gate, and then design scheduling strategies to exploit this ability and achieve suppression across the whole circuit. A main advantage of such co-optimization is that it does not require special hardware support. Besides, we implement our approach as a general framework that is compatible with different pulse optimization methods. We have conducted extensive evaluations by simulation and on a real quantum computer. Simulation results show that our proposal can improve the fidelity of quantum computing on 4∼12 qubits by up to 81× (11× on average). Ramsey experiments on a real quantum computer also demonstrate that our method can eliminate the effect of ZZ crosstalk to a great extent. @InProceedings{ASPLOS22p499, author = {Lei Xie and Jidong Zhai and ZhenXing Zhang and Jonathan Allcock and Shengyu Zhang and Yi-Cong Zheng}, title = {Suppressing ZZ Crosstalk of Quantum Computers through Pulse and Scheduling Co-Optimization}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {499--513}, doi = {10.1145/3503222.3507761}, year = {2022}, } Publisher's Version
	Zhang, Zhihui	ASPLOS '22: "VELTAIR: Towards High-Performance ..." VELTAIR: Towards High-Performance Multi-tenant Deep Learning Services via Adaptive Compilation and Scheduling Zihan Liu, Jingwen Leng, Zhihui Zhang, Quan Chen, Chao Li, and Minyi Guo (Shanghai Jiao Tong University, China; Shanghai Qi Zhi Institute, China) Deep learning (DL) models have achieved great success in many application domains. As such, many industrial companies such as Google and Facebook have acknowledged the importance of multi-tenant DL services. Although the multi-tenant service has been studied in conventional workloads, it is not been deeply studied on deep learning service, especially on general-purpose hardware. In this work, we systematically analyze the opportunities and challenges of providing multi-tenant deep learning services on the general-purpose CPU architecture from the aspects of scheduling granularity and code generation. We propose an adaptive granularity scheduling scheme to both guarantee resource usage efficiency and reduce the scheduling conflict rate. We also propose an adaptive compilation strategy, by which we can dynamically and intelligently pick a program with proper exclusive and shared resource usage to reduce overall interference-induced performance loss. Compared to the existing works, our design can serve more requests under the same QoS target in various scenarios (e.g., +71%, +62%, +45% for light, medium, and heavy workloads, respectively), and reduce the averaged query latency by 50%. @InProceedings{ASPLOS22p388, author = {Zihan Liu and Jingwen Leng and Zhihui Zhang and Quan Chen and Chao Li and Minyi Guo}, title = {VELTAIR: Towards High-Performance Multi-tenant Deep Learning Services via Adaptive Compilation and Scheduling}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {388--401}, doi = {10.1145/3503222.3507752}, year = {2022}, } Publisher's Version
	Zhao, Laiping	ASPLOS '22: "INFless: A Native Serverless ..." INFless: A Native Serverless System for Low-Latency, High-Throughput Inference Yanan Yang, Laiping Zhao, Yiming Li, Huanyu Zhang, Jie Li, Mingyang Zhao, Xingzhen Chen, and Keqiu Li (Tianjin University, China; 58.com, China) Modern websites increasingly rely on machine learning (ML) to improve their business efficiency. Developing and maintaining ML services incurs high costs for developers. Although serverless systems are a promising solution to reduce costs, we find that the current general purpose serverless systems cannot meet the low latency, high throughput demands of ML services. While simply ”patching” general serverless systems does not resolve the problem completely, we propose that such a system should natively combine the features of inference with a serverless paradigm. We present INFless, the first ML domain-specific serverless platform. It provides a unified, heterogeneous resource abstraction between CPU and accelerators, and achieves high throughput using built-in batching and non-uniform scaling mechanisms. It also supports low latency through coordinated management of batch queuing time, execution time and coldstart rate. We evaluate INFless using both a local cluster testbed and a large-scale simulation. Experimental results show that INFless outperforms state-of-the-art systems by 2×-5× on system throughput, meeting the latency goals of ML services. @InProceedings{ASPLOS22p768, author = {Yanan Yang and Laiping Zhao and Yiming Li and Huanyu Zhang and Jie Li and Mingyang Zhao and Xingzhen Chen and Keqiu Li}, title = {INFless: A Native Serverless System for Low-Latency, High-Throughput Inference}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {768--781}, doi = {10.1145/3503222.3507709}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available
	Zhao, Mark	ASPLOS '22: "ShEF: Shielded Enclaves for ..." ShEF: Shielded Enclaves for Cloud FPGAs Mark Zhao, Mingyu Gao, and Christos Kozyrakis (Stanford University, USA; Tsinghua University, China) FPGAs are now used in public clouds to accelerate a wide range of applications, including many that operate on sensitive data such as financial and medical records. We present ShEF, a trusted execution environment (TEE) for cloud-based reconfigurable accelerators. ShEF is independent from CPU-based TEEs and allows secure execution under a threat model where the adversary can control all software running on the CPU connected to the FPGA, has physical access to the FPGA, and can compromise the FPGA interface logic of the cloud provider. ShEF provides a secure boot and remote attestation process that relies solely on existing FPGA mechanisms for root of trust. It also includes a Shield component that provides secure access to data while the accelerator is in use. The Shield is highly customizable and extensible, allowing users to craft a bespoke security solution that fits their accelerator's memory access patterns, bandwidth, and security requirements at minimum performance and area overheads. We describe a prototype implementation of ShEF for existing cloud FPGAs, map ShEF to a performant and secure storage application, and measure the performance benefits of customizable security using five additional accelerators. @InProceedings{ASPLOS22p1070, author = {Mark Zhao and Mingyu Gao and Christos Kozyrakis}, title = {ShEF: Shielded Enclaves for Cloud FPGAs}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {1070--1085}, doi = {10.1145/3503222.3507733}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional
	Zhao, Mingyang	ASPLOS '22: "INFless: A Native Serverless ..." INFless: A Native Serverless System for Low-Latency, High-Throughput Inference Yanan Yang, Laiping Zhao, Yiming Li, Huanyu Zhang, Jie Li, Mingyang Zhao, Xingzhen Chen, and Keqiu Li (Tianjin University, China; 58.com, China) Modern websites increasingly rely on machine learning (ML) to improve their business efficiency. Developing and maintaining ML services incurs high costs for developers. Although serverless systems are a promising solution to reduce costs, we find that the current general purpose serverless systems cannot meet the low latency, high throughput demands of ML services. While simply ”patching” general serverless systems does not resolve the problem completely, we propose that such a system should natively combine the features of inference with a serverless paradigm. We present INFless, the first ML domain-specific serverless platform. It provides a unified, heterogeneous resource abstraction between CPU and accelerators, and achieves high throughput using built-in batching and non-uniform scaling mechanisms. It also supports low latency through coordinated management of batch queuing time, execution time and coldstart rate. We evaluate INFless using both a local cluster testbed and a large-scale simulation. Experimental results show that INFless outperforms state-of-the-art systems by 2×-5× on system throughput, meeting the latency goals of ML services. @InProceedings{ASPLOS22p768, author = {Yanan Yang and Laiping Zhao and Yiming Li and Huanyu Zhang and Jie Li and Mingyang Zhao and Xingzhen Chen and Keqiu Li}, title = {INFless: A Native Serverless System for Low-Latency, High-Throughput Inference}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {768--781}, doi = {10.1145/3503222.3507709}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available
	Zhao, Pengzhan	ASPLOS '22: "AStitch: Enabling a New Multi-dimensional ..." AStitch: Enabling a New Multi-dimensional Optimization Space for Memory-Intensive ML Training and Inference on Modern SIMT Architectures Zhen Zheng, Xuanda Yang, Pengzhan Zhao, Guoping Long, Kai Zhu, Feiwen Zhu, Wenyi Zhao, Xiaoyong Liu, Jun Yang, Jidong Zhai, Shuaiwen Leon Song, and Wei Lin (Alibaba Group, China; Tsinghua University, China; University of Sydney, Australia) This work reveals that memory-intensive computation is a rising performance-critical factor in recent machine learning models. Due to a unique set of new challenges, existing ML optimizing compilers cannot perform efficient fusion under complex two-level dependencies combined with just-in-time demand. They face the dilemma of either performing costly fusion due to heavy redundant computation, or skipping fusion which results in massive number of kernels. Furthermore, they often suffer from low parallelism due to the lack of support for real-world production workloads with irregular tensor shapes. To address these rising challenges, we propose AStitch, a machine learning optimizing compiler that opens a new multi-dimensional optimization space for memory-intensive ML computations. It systematically abstracts four operator-stitching schemes while considering multi-dimensional optimization objectives, tackles complex computation graph dependencies with novel hierarchical data reuse, and efficiently processes various tensor shapes via adaptive thread mapping. Finally, AStitch provides just-in-time support incorporating our proposed optimizations for both ML training and inference. Although AStitch serves as a stand-alone compiler engine that is portable to any version of TensorFlow, its basic ideas can be generally applied to other ML frameworks and optimization compilers. Experimental results show that AStitch can achieve an average of 1.84x speedup (up to 2.73x) over the state-of-the-art Google's XLA solution across five production workloads. We also deploy AStitch onto a production cluster for ML workloads with thousands of GPUs. The system has been in operation for more than 10 months and saves about 20,000 GPU hours for 70,000 tasks per week. @InProceedings{ASPLOS22p359, author = {Zhen Zheng and Xuanda Yang and Pengzhan Zhao and Guoping Long and Kai Zhu and Feiwen Zhu and Wenyi Zhao and Xiaoyong Liu and Jun Yang and Jidong Zhai and Shuaiwen Leon Song and Wei Lin}, title = {AStitch: Enabling a New Multi-dimensional Optimization Space for Memory-Intensive ML Training and Inference on Modern SIMT Architectures}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {359--373}, doi = {10.1145/3503222.3507723}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Zhao, Shixiong	ASPLOS '22: "NASPipe: High Performance ..." NASPipe: High Performance and Reproducible Pipeline Parallel Supernet Training via Causal Synchronous Parallelism Shixiong Zhao, Fanxin Li, Xusheng Chen, Tianxiang Shen, Li Chen, Sen Wang, Nicholas Zhang, Cheng Li, and Heming Cui (University of Hong Kong, China; Huawei Technologies, China; University of Science and Technology of China, China) Supernet training, a prevalent and important paradigm in Neural Architecture Search, embeds the whole DNN architecture search space into one monolithic supernet, iteratively activates a subset of the supernet (i.e., a subnet) for fitting each batch of data, and searches a high-quality subnet which meets specific requirements. Although training subnets in parallel on multiple GPUs is desirable for acceleration, there inherently exists a race hazard that concurrent subnets may access the same DNN layers. Existing systems support neither efficiently parallelizing subnets’ training executions, nor resolving the race hazard deterministically, leading to unreproducible training procedures and potentiallly non-trivial accuracy loss. We present NASPipe, the first high-performance and reproducible distributed supernet training system via causal synchronous parallel (CSP) pipeline scheduling abstraction: NASPipe partitions a supernet across GPUs and concurrently executes multiple generated sub-tasks (subnets) in a pipelined manner; meanwhile, it oversees the correlations between the subnets and deterministically resolves any causal dependency caused by subnets’ layer sharing. To obtain high performance, NASPipe’s CSP scheduler exploits the fact that the larger a supernet spans, the fewer dependencies manifest between chronologically close subnets; therefore, it aggressively schedules the subnets with larger chronological orders into execution, only if they are not causally dependent on unfinished precedent subnets. Moreover, to relieve the excessive GPU memory burden for holding the whole supernet’s parameters, NASPipe uses a context switch technique that stashes the whole supernet in CPU memory, precisely predicts the subnets’ schedule, and pre-fetches/evicts a subnet before/after its execution. The evaluation shows that NASPipe is the only system that retains supernet training reproducibility, while achieving a comparable and even higher performance (up to 7.8X) compared to three recent pipeline training systems (e.g., GPipe). @InProceedings{ASPLOS22p374, author = {Shixiong Zhao and Fanxin Li and Xusheng Chen and Tianxiang Shen and Li Chen and Sen Wang and Nicholas Zhang and Cheng Li and Heming Cui}, title = {NASPipe: High Performance and Reproducible Pipeline Parallel Supernet Training via Causal Synchronous Parallelism}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {374--387}, doi = {10.1145/3503222.3507735}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Zhao, Wenyi	ASPLOS '22: "AStitch: Enabling a New Multi-dimensional ..." AStitch: Enabling a New Multi-dimensional Optimization Space for Memory-Intensive ML Training and Inference on Modern SIMT Architectures Zhen Zheng, Xuanda Yang, Pengzhan Zhao, Guoping Long, Kai Zhu, Feiwen Zhu, Wenyi Zhao, Xiaoyong Liu, Jun Yang, Jidong Zhai, Shuaiwen Leon Song, and Wei Lin (Alibaba Group, China; Tsinghua University, China; University of Sydney, Australia) This work reveals that memory-intensive computation is a rising performance-critical factor in recent machine learning models. Due to a unique set of new challenges, existing ML optimizing compilers cannot perform efficient fusion under complex two-level dependencies combined with just-in-time demand. They face the dilemma of either performing costly fusion due to heavy redundant computation, or skipping fusion which results in massive number of kernels. Furthermore, they often suffer from low parallelism due to the lack of support for real-world production workloads with irregular tensor shapes. To address these rising challenges, we propose AStitch, a machine learning optimizing compiler that opens a new multi-dimensional optimization space for memory-intensive ML computations. It systematically abstracts four operator-stitching schemes while considering multi-dimensional optimization objectives, tackles complex computation graph dependencies with novel hierarchical data reuse, and efficiently processes various tensor shapes via adaptive thread mapping. Finally, AStitch provides just-in-time support incorporating our proposed optimizations for both ML training and inference. Although AStitch serves as a stand-alone compiler engine that is portable to any version of TensorFlow, its basic ideas can be generally applied to other ML frameworks and optimization compilers. Experimental results show that AStitch can achieve an average of 1.84x speedup (up to 2.73x) over the state-of-the-art Google's XLA solution across five production workloads. We also deploy AStitch onto a production cluster for ML workloads with thousands of GPUs. The system has been in operation for more than 10 months and saves about 20,000 GPU hours for 70,000 tasks per week. @InProceedings{ASPLOS22p359, author = {Zhen Zheng and Xuanda Yang and Pengzhan Zhao and Guoping Long and Kai Zhu and Feiwen Zhu and Wenyi Zhao and Xiaoyong Liu and Jun Yang and Jidong Zhai and Shuaiwen Leon Song and Wei Lin}, title = {AStitch: Enabling a New Multi-dimensional Optimization Space for Memory-Intensive ML Training and Inference on Modern SIMT Architectures}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {359--373}, doi = {10.1145/3503222.3507723}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Zhao, Zhijia	ASPLOS '22: "JSONSki: Streaming Semi-structured ..." JSONSki: Streaming Semi-structured Data with Bit-Parallel Fast-Forwarding Lin Jiang and Zhijia Zhao (University of California at Riverside, USA) Semi-structured data, such as JSON, are fundamental to the Web and document data stores. Streaming analytics on semi-structured data combines parsing and query evaluation into one pass to avoid generating parse trees. Though promising, its conventional design requires to parse the data stream in detail character by character, which limits the efficiency of streaming analytics. This work reveals a wide range of opportunities to fast-forward the streaming over certain data substructures irrelevant to the query evaluation. However, identifying these substructures itself may need detailed parsing. To resolve this dilemma, this work designs a highly bit-parallel solution that intensively utilizes bitwise and SIMD operations to identify the irrelevant substructures during the streaming. It includes a new streaming model—recursive-descent streaming, for an easy adoption of fast-forward optimizations, a concept—structural intervals, for partitioning the data stream, and a group of bit-parallel algorithms implementing various fast-forward cases. The solution is implemented as a JSON streaming framework, called JSONSki. It offers a set of APIs that can be invoked during the streaming to dynamically fast-forward over different cases of irrelevant substructures. Evaluation using real-world datasets and standard path queries shows that JSONSki can achieve significant speedups over the state-of-the-art JSON processing tools while taking a minimum memory footprint. @InProceedings{ASPLOS22p200, author = {Lin Jiang and Zhijia Zhao}, title = {JSONSki: Streaming Semi-structured Data with Bit-Parallel Fast-Forwarding}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {200--211}, doi = {10.1145/3503222.3507719}, year = {2022}, } Publisher's Version
	Zhao, Zirui Neil	ASPLOS '22: "Pinned Loads: Taming Speculative ..." Pinned Loads: Taming Speculative Loads in Secure Processors Zirui Neil Zhao, Houxiang Ji, Adam Morrison, Darko Marinov, and Josep Torrellas (University of Illinois at Urbana-Champaign, USA; Tel Aviv University, Israel) In security frameworks for speculative execution, an instruction is said to reach its Visibility Point (VP) when it is no longer vulnerable to pipeline squashes. Before a potentially leaky instruction reaches its VP, it has to stall—unless a defense scheme such as invisible speculation provides protection. Unfortunately, either stalling or protecting the execution of pre-VP instructions typically has a performance cost. One way to attain low-overhead safe execution is to develop techniques that speed-up the advance of the VP from older to younger instructions. In this paper, we propose one such technique. We find that the progress of the VP for loads is mostly impeded by waiting until no memory consistency violations (MCVs) are possible. Hence, our technique, called , tries to make loads invulnerable to MCVs as early as possible—a process we call pinning the loads in the pipeline. The result is faster VP progress and a reduction in the execution overhead of defense schemes. In this paper, we describe the hardware needed by , and two possible designs with different tradeoffs between hardware requirements and performance. Our evaluation shows that is very effective: extending three popular defense schemes against speculative execution attacks with reduces their average execution overhead on SPEC17 and on SPLASH2/PARSEC applications by about 50%. For example, on SPEC17, the execution overhead of the three defense schemes decreases from to , from to , and from to . @InProceedings{ASPLOS22p314, author = {Zirui Neil Zhao and Houxiang Ji and Adam Morrison and Darko Marinov and Josep Torrellas}, title = {Pinned Loads: Taming Speculative Loads in Secure Processors}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {314--328}, doi = {10.1145/3503222.3507724}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Zheng, Ningxin	ASPLOS '22: "Astraea: Towards QoS-Aware ..." Astraea: Towards QoS-Aware and Resource-Efficient Multi-stage GPU Services Wei Zhang, Quan Chen, Kaihua Fu, Ningxin Zheng, Zhiyi Huang, Jingwen Leng, and Minyi Guo (Shanghai Jiao Tong University, China; Microsoft Research, China; University of Otago, New Zealand) Multi-stage user-facing applications on GPUs are widely-used nowa- days, and are often implemented to be microservices. Prior re- search works are not applicable to ensuring QoS of GPU-based microservices due to the different communication patterns and shared resource contentions. We propose Astraea to manage GPU microservices considering the above factors. In Astraea, a microser- vice deployment policy is used to maximize the supported peak service load while ensuring the required QoS. To adaptively switch the communication methods between microservices according to different deployments, we propose an auto-scaling GPU communi- cation framework. The framework automatically scales based on the currently used hardware topology and microservice location, and adopts global memory-based techniques to reduce intra-GPU communication. Astraea increases the supported peak load by up to 82.3% while achieving the desired 99%-ile latency target compared with state-of-the-art solutions. @InProceedings{ASPLOS22p570, author = {Wei Zhang and Quan Chen and Kaihua Fu and Ningxin Zheng and Zhiyi Huang and Jingwen Leng and Minyi Guo}, title = {Astraea: Towards QoS-Aware and Resource-Efficient Multi-stage GPU Services}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {570--582}, doi = {10.1145/3503222.3507721}, year = {2022}, } Publisher's Version
	Zheng, Wenli	ASPLOS '22: "FaaSFlow: Enable Efficient ..." FaaSFlow: Enable Efficient Workflow Execution for Function-as-a-Service Zijun Li, Yushi Liu, Linsong Guo, Quan Chen, Jiagan Cheng, Wenli Zheng, and Minyi Guo (Shanghai Jiao Tong University, China) Serverless computing (Function-as-a-Service) provides fine-grain resource sharing by running functions (or Lambdas) in containers. Data-dependent functions are required to be invoked following a pre-defined logic, which is known as serverless workflows. However, our investigation shows that the traditional master-worker based workflow execution architecture performs poorly in serverless context. One significant overhead results from the master-side workflow schedule pattern, with which the functions are triggered in the master node and assigned to worker nodes for execution. Besides, the data movement between workers also reduces the throughput. To this end, we present a worker-side workflow schedule pattern for serverless workflow execution. Following the design, we implement FaaSFlow to enable efficient workflow execution in the serverless context. Besides, we propose an adaptive storage library FaaStore that enables fast data transfer between functions on the same node without through the database. Experiment results show that FaaSFlow effectively mitigates the workflow scheduling overhead by 74.6% on average and data transmission overhead by 95% at most. When the network bandwidth fluctuates, FaaSFlow-FaaStore reduces the throughput degradation by 23.0%, and is able to multiply the utilization of network bandwidth by 1.5X-4X. @InProceedings{ASPLOS22p782, author = {Zijun Li and Yushi Liu and Linsong Guo and Quan Chen and Jiagan Cheng and Wenli Zheng and Minyi Guo}, title = {FaaSFlow: Enable Efficient Workflow Execution for Function-as-a-Service}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {782--796}, doi = {10.1145/3503222.3507717}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Zheng, Yi-Cong	ASPLOS '22: "Suppressing ZZ Crosstalk of ..." Suppressing ZZ Crosstalk of Quantum Computers through Pulse and Scheduling Co-Optimization Lei Xie, Jidong Zhai, ZhenXing Zhang, Jonathan Allcock, Shengyu Zhang, and Yi-Cong Zheng (Tsinghua University, China; Tencent Quantum Laboratory, China) Noise is a significant obstacle to quantum computing, and ZZ crosstalk is one of the most destructive types of noise affecting superconducting qubits. Previous approaches to suppressing ZZ crosstalk have mainly relied on specific chip design that can complicate chip fabrication and aggravate decoherence. To some extent, special chip design can be avoided by relying on pulse optimization to suppress ZZ crosstalk. However, existing approaches are non-scalable, as their required time and memory grow exponentially with the number of qubits involved. To address the above problems, we propose a scalable approach by co-optimizing pulses and scheduling. We optimize pulses to offer an ability to suppress ZZ crosstalk surrounding a gate, and then design scheduling strategies to exploit this ability and achieve suppression across the whole circuit. A main advantage of such co-optimization is that it does not require special hardware support. Besides, we implement our approach as a general framework that is compatible with different pulse optimization methods. We have conducted extensive evaluations by simulation and on a real quantum computer. Simulation results show that our proposal can improve the fidelity of quantum computing on 4∼12 qubits by up to 81× (11× on average). Ramsey experiments on a real quantum computer also demonstrate that our method can eliminate the effect of ZZ crosstalk to a great extent. @InProceedings{ASPLOS22p499, author = {Lei Xie and Jidong Zhai and ZhenXing Zhang and Jonathan Allcock and Shengyu Zhang and Yi-Cong Zheng}, title = {Suppressing ZZ Crosstalk of Quantum Computers through Pulse and Scheduling Co-Optimization}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {499--513}, doi = {10.1145/3503222.3507761}, year = {2022}, } Publisher's Version
	Zheng, Zhen	ASPLOS '22: "AStitch: Enabling a New Multi-dimensional ..." AStitch: Enabling a New Multi-dimensional Optimization Space for Memory-Intensive ML Training and Inference on Modern SIMT Architectures Zhen Zheng, Xuanda Yang, Pengzhan Zhao, Guoping Long, Kai Zhu, Feiwen Zhu, Wenyi Zhao, Xiaoyong Liu, Jun Yang, Jidong Zhai, Shuaiwen Leon Song, and Wei Lin (Alibaba Group, China; Tsinghua University, China; University of Sydney, Australia) This work reveals that memory-intensive computation is a rising performance-critical factor in recent machine learning models. Due to a unique set of new challenges, existing ML optimizing compilers cannot perform efficient fusion under complex two-level dependencies combined with just-in-time demand. They face the dilemma of either performing costly fusion due to heavy redundant computation, or skipping fusion which results in massive number of kernels. Furthermore, they often suffer from low parallelism due to the lack of support for real-world production workloads with irregular tensor shapes. To address these rising challenges, we propose AStitch, a machine learning optimizing compiler that opens a new multi-dimensional optimization space for memory-intensive ML computations. It systematically abstracts four operator-stitching schemes while considering multi-dimensional optimization objectives, tackles complex computation graph dependencies with novel hierarchical data reuse, and efficiently processes various tensor shapes via adaptive thread mapping. Finally, AStitch provides just-in-time support incorporating our proposed optimizations for both ML training and inference. Although AStitch serves as a stand-alone compiler engine that is portable to any version of TensorFlow, its basic ideas can be generally applied to other ML frameworks and optimization compilers. Experimental results show that AStitch can achieve an average of 1.84x speedup (up to 2.73x) over the state-of-the-art Google's XLA solution across five production workloads. We also deploy AStitch onto a production cluster for ML workloads with thousands of GPUs. The system has been in operation for more than 10 months and saves about 20,000 GPU hours for 70,000 tasks per week. @InProceedings{ASPLOS22p359, author = {Zhen Zheng and Xuanda Yang and Pengzhan Zhao and Guoping Long and Kai Zhu and Feiwen Zhu and Wenyi Zhao and Xiaoyong Liu and Jun Yang and Jidong Zhai and Shuaiwen Leon Song and Wei Lin}, title = {AStitch: Enabling a New Multi-dimensional Optimization Space for Memory-Intensive ML Training and Inference on Modern SIMT Architectures}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {359--373}, doi = {10.1145/3503222.3507723}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Zhong, Ming	ASPLOS '22: "EXAMINER: Automatically Locating ..." EXAMINER: Automatically Locating Inconsistent Instructions between Real Devices and CPU Emulators for ARM Muhui Jiang, Tianyi Xu, Yajin Zhou, Yufeng Hu, Ming Zhong, Lei Wu, Xiapu Luo, and Kui Ren (Hong Kong Polytechnic University, China; Zhejiang University, China) Emulators are widely used to build dynamic analysis frameworks due to its fine-grained tracing capability, full system monitoring functionality, and scalability of running on different operating systems and architectures. However, whether emulators are consistent with real devices is unknown. To understand this problem, we aim to automatically locate inconsistent instructions, which behave differently between emulators and real devices. We target the ARM architecture, which provides machine-readable specifications. Based on the specification, we propose a sufficient test case generator by designing and implementing the first symbolic execution engine for the ARM architecture specification language (ASL). We generate 2,774,649 representative instruction streams and conduct differential testing between four ARM real devices in different architecture versions (i.e., ARMv5, ARMv6, ARMv7, and ARMv8) and three state-of-the-art emulators (i.e., QEMU, Unicorn, and Angr). We locate a huge number of inconsistent instruction streams (171,858 for QEMU, 223,264 for unicorn, and 120,169 for Angr). We find that undefined implementation in ARM manual and bugs of emulators are the major causes of inconsistencies. Furthermore, we discover 12 bugs, which influence commonly used instructions (e.g., BLX). With the inconsistent instructions, we build three security applications and demonstrate the capability of these instructions on detecting emulators, anti-emulation, and anti-fuzzing. @InProceedings{ASPLOS22p846, author = {Muhui Jiang and Tianyi Xu and Yajin Zhou and Yufeng Hu and Ming Zhong and Lei Wu and Xiapu Luo and Kui Ren}, title = {EXAMINER: Automatically Locating Inconsistent Instructions between Real Devices and CPU Emulators for ARM}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {846--858}, doi = {10.1145/3503222.3507736}, year = {2022}, } Publisher's Version
	Zhou, Keren	ASPLOS '22: "ValueExpert: Exploring Value ..." ValueExpert: Exploring Value Patterns in GPU-Accelerated Applications Keren Zhou, Yueming Hao, John Mellor-Crummey, Xiaozhu Meng, and Xu Liu (Rice University, USA; North Carolina State University, USA) General-purpose GPUs have become common in modern computing systems to accelerate applications in many domains, including machine learning, high-performance computing, and autonomous driving. However, inefficiencies abound in GPU-accelerated applications, which prevent them from obtaining bare-metal performance. Performance tools play an important role in understanding performance inefficiencies in complex code bases. Many GPU performance tools pinpoint time-consuming code and provide high-level performance insights but overlook one important performance issue---value-related inefficiencies, which exist in many GPU code bases. In this paper, we present ValueExpert, a novel tool to pinpoint value-related inefficiencies in GPU applications. ValueExpert monitors application execution to capture values produced and used by each load and store operation in GPU kernels, recognizes multiple value patterns, and provides intuitive optimization guidance. We address systemic challenges in collecting, maintaining, and analyzing voluminous performance data from many GPU threads to make ValueExpert applicable to complex applications. We evaluate ValueExpert on a wide range of well-tuned benchmarks and applications, including PyTorch, Darknet, LAMMPS, Castro, and many others. ValueExpert is able to identify previously unknown performance issues and provide suggestions for nontrivial performance improvements with typically less than five lines of code changes. We verify our optimizations with application developers and upstream fixes to their repositories. @InProceedings{ASPLOS22p171, author = {Keren Zhou and Yueming Hao and John Mellor-Crummey and Xiaozhu Meng and Xu Liu}, title = {ValueExpert: Exploring Value Patterns in GPU-Accelerated Applications}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {171--185}, doi = {10.1145/3503222.3507708}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Zhou, Yajin	ASPLOS '22: "EXAMINER: Automatically Locating ..." EXAMINER: Automatically Locating Inconsistent Instructions between Real Devices and CPU Emulators for ARM Muhui Jiang, Tianyi Xu, Yajin Zhou, Yufeng Hu, Ming Zhong, Lei Wu, Xiapu Luo, and Kui Ren (Hong Kong Polytechnic University, China; Zhejiang University, China) Emulators are widely used to build dynamic analysis frameworks due to its fine-grained tracing capability, full system monitoring functionality, and scalability of running on different operating systems and architectures. However, whether emulators are consistent with real devices is unknown. To understand this problem, we aim to automatically locate inconsistent instructions, which behave differently between emulators and real devices. We target the ARM architecture, which provides machine-readable specifications. Based on the specification, we propose a sufficient test case generator by designing and implementing the first symbolic execution engine for the ARM architecture specification language (ASL). We generate 2,774,649 representative instruction streams and conduct differential testing between four ARM real devices in different architecture versions (i.e., ARMv5, ARMv6, ARMv7, and ARMv8) and three state-of-the-art emulators (i.e., QEMU, Unicorn, and Angr). We locate a huge number of inconsistent instruction streams (171,858 for QEMU, 223,264 for unicorn, and 120,169 for Angr). We find that undefined implementation in ARM manual and bugs of emulators are the major causes of inconsistencies. Furthermore, we discover 12 bugs, which influence commonly used instructions (e.g., BLX). With the inconsistent instructions, we build three security applications and demonstrate the capability of these instructions on detecting emulators, anti-emulation, and anti-fuzzing. @InProceedings{ASPLOS22p846, author = {Muhui Jiang and Tianyi Xu and Yajin Zhou and Yufeng Hu and Ming Zhong and Lei Wu and Xiapu Luo and Kui Ren}, title = {EXAMINER: Automatically Locating Inconsistent Instructions between Real Devices and CPU Emulators for ARM}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {846--858}, doi = {10.1145/3503222.3507736}, year = {2022}, } Publisher's Version
	Zhu, Feiwen	ASPLOS '22: "AStitch: Enabling a New Multi-dimensional ..." AStitch: Enabling a New Multi-dimensional Optimization Space for Memory-Intensive ML Training and Inference on Modern SIMT Architectures Zhen Zheng, Xuanda Yang, Pengzhan Zhao, Guoping Long, Kai Zhu, Feiwen Zhu, Wenyi Zhao, Xiaoyong Liu, Jun Yang, Jidong Zhai, Shuaiwen Leon Song, and Wei Lin (Alibaba Group, China; Tsinghua University, China; University of Sydney, Australia) This work reveals that memory-intensive computation is a rising performance-critical factor in recent machine learning models. Due to a unique set of new challenges, existing ML optimizing compilers cannot perform efficient fusion under complex two-level dependencies combined with just-in-time demand. They face the dilemma of either performing costly fusion due to heavy redundant computation, or skipping fusion which results in massive number of kernels. Furthermore, they often suffer from low parallelism due to the lack of support for real-world production workloads with irregular tensor shapes. To address these rising challenges, we propose AStitch, a machine learning optimizing compiler that opens a new multi-dimensional optimization space for memory-intensive ML computations. It systematically abstracts four operator-stitching schemes while considering multi-dimensional optimization objectives, tackles complex computation graph dependencies with novel hierarchical data reuse, and efficiently processes various tensor shapes via adaptive thread mapping. Finally, AStitch provides just-in-time support incorporating our proposed optimizations for both ML training and inference. Although AStitch serves as a stand-alone compiler engine that is portable to any version of TensorFlow, its basic ideas can be generally applied to other ML frameworks and optimization compilers. Experimental results show that AStitch can achieve an average of 1.84x speedup (up to 2.73x) over the state-of-the-art Google's XLA solution across five production workloads. We also deploy AStitch onto a production cluster for ML workloads with thousands of GPUs. The system has been in operation for more than 10 months and saves about 20,000 GPU hours for 70,000 tasks per week. @InProceedings{ASPLOS22p359, author = {Zhen Zheng and Xuanda Yang and Pengzhan Zhao and Guoping Long and Kai Zhu and Feiwen Zhu and Wenyi Zhao and Xiaoyong Liu and Jun Yang and Jidong Zhai and Shuaiwen Leon Song and Wei Lin}, title = {AStitch: Enabling a New Multi-dimensional Optimization Space for Memory-Intensive ML Training and Inference on Modern SIMT Architectures}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {359--373}, doi = {10.1145/3503222.3507723}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Zhu, Kai	ASPLOS '22: "AStitch: Enabling a New Multi-dimensional ..." AStitch: Enabling a New Multi-dimensional Optimization Space for Memory-Intensive ML Training and Inference on Modern SIMT Architectures Zhen Zheng, Xuanda Yang, Pengzhan Zhao, Guoping Long, Kai Zhu, Feiwen Zhu, Wenyi Zhao, Xiaoyong Liu, Jun Yang, Jidong Zhai, Shuaiwen Leon Song, and Wei Lin (Alibaba Group, China; Tsinghua University, China; University of Sydney, Australia) This work reveals that memory-intensive computation is a rising performance-critical factor in recent machine learning models. Due to a unique set of new challenges, existing ML optimizing compilers cannot perform efficient fusion under complex two-level dependencies combined with just-in-time demand. They face the dilemma of either performing costly fusion due to heavy redundant computation, or skipping fusion which results in massive number of kernels. Furthermore, they often suffer from low parallelism due to the lack of support for real-world production workloads with irregular tensor shapes. To address these rising challenges, we propose AStitch, a machine learning optimizing compiler that opens a new multi-dimensional optimization space for memory-intensive ML computations. It systematically abstracts four operator-stitching schemes while considering multi-dimensional optimization objectives, tackles complex computation graph dependencies with novel hierarchical data reuse, and efficiently processes various tensor shapes via adaptive thread mapping. Finally, AStitch provides just-in-time support incorporating our proposed optimizations for both ML training and inference. Although AStitch serves as a stand-alone compiler engine that is portable to any version of TensorFlow, its basic ideas can be generally applied to other ML frameworks and optimization compilers. Experimental results show that AStitch can achieve an average of 1.84x speedup (up to 2.73x) over the state-of-the-art Google's XLA solution across five production workloads. We also deploy AStitch onto a production cluster for ML workloads with thousands of GPUs. The system has been in operation for more than 10 months and saves about 20,000 GPU hours for 70,000 tasks per week. @InProceedings{ASPLOS22p359, author = {Zhen Zheng and Xuanda Yang and Pengzhan Zhao and Guoping Long and Kai Zhu and Feiwen Zhu and Wenyi Zhao and Xiaoyong Liu and Jun Yang and Jidong Zhai and Shuaiwen Leon Song and Wei Lin}, title = {AStitch: Enabling a New Multi-dimensional Optimization Space for Memory-Intensive ML Training and Inference on Modern SIMT Architectures}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {359--373}, doi = {10.1145/3503222.3507723}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced
	Zhu, Xinhui	ASPLOS '22: "CoolEdge: Hotspot-Relievable ..." CoolEdge: Hotspot-Relievable Warm Water Cooling for Energy-Efficient Edge Datacenters Qiangyu Pei, Shutong Chen, Qixia Zhang, Xinhui Zhu, Fangming Liu, Ziyang Jia, Yishuo Wang, and Yongjie Yuan (Huazhong University of Science and Technology, China) As the computing frontier drifts to the edge, edge datacenters play a crucial role in supporting various real-time applications. Different from cloud datacenters, the requirements of proximity to end-users, high density, and heterogeneity, present new challenges to cool the edge datacenters efficiently. Although warm water cooling has become a promising cooling technique for this infrastructure, the one-size-fits-all cooling control would lower the cooling efficiency considerably because of the severe thermal imbalance across servers, hardware, and even inside one hardware component in an edge datacenter. In this work, we propose CoolEdge, a hotspot-relievable warm water cooling system for improving the cooling efficiency and saving costs of edge datacenters. Specifically, through the elaborate design of water circulations, CoolEdge can dynamically adjust the water temperature and flow rate for each heterogeneous hardware component to eliminate the hardware-level hotspots. By redesigning cold plates, CoolEdge can quickly disperse the chip-level hotspots without manual intervention. We further quantify the power saving achieved by the warm water cooling theoretically, and propose a custom-designed cooling solution to decide an appropriate water temperature and flow rate periodically. Based on a hardware prototype and real-world traces from SURFsara, the evaluation results show that CoolEdge reduces the cooling energy by 81.81% and 71.92%, respectively, compared with conventional and state-of-the-art water cooling systems. @InProceedings{ASPLOS22p814, author = {Qiangyu Pei and Shutong Chen and Qixia Zhang and Xinhui Zhu and Fangming Liu and Ziyang Jia and Yishuo Wang and Yongjie Yuan}, title = {CoolEdge: Hotspot-Relievable Warm Water Cooling for Energy-Efficient Edge Datacenters}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {814--829}, doi = {10.1145/3503222.3507713}, year = {2022}, } Publisher's Version
	Zuo, Gefei	ASPLOS '22: "Debugging in the Brave New ..." Debugging in the Brave New World of Reconfigurable Hardware Jiacheng Ma, Gefei Zuo, Kevin Loughlin, Haoyang Zhang, Andrew Quinn, and Baris Kasikci (University of Michigan, USA; University of California at Santa Cruz, USA) Software and hardware development cycles have traditionally been quite distinct. Software allows post-deployment patches, which leads to a rapid development cycle. In contrast, hardware bugs that are found after fabrication are extremely costly to fix (and sometimes even unfixable), so the traditional hardware development cycle involves massive investment in extensive simulation and formal verification. Reconfigurable hardware, such as a Field Programmable Gate Array (FPGA), promises to propel hardware development towards an agile software-like development approach, since it enables a hardware developer to patch bugs that are detected during on-chip testing or in production. Unfortunately, FPGA programmers lack bug localization tools amenable to this rapid development cycle, since past tools mainly find bugs via simulation and verification. To develop hardware bug localization tools for a rapid development cycle, a thorough understanding of the symptoms, root causes, and fixes of hardware bugs is needed. In this paper, we first study bugs in existing FPGA designs and produce a testbed of reliably-reproducible bugs. We classify the bugs according to their intrinsic properties, symptoms, and root causes. We demonstrate that many hardware bugs are comparable to software bug counterparts, and would benefit from similar techniques for bug diagnosis and repair. Based upon our findings, we build a novel collection of hybrid static/dynamic program analysis and monitoring tools for debugging FPGA designs, showing that our tools enable a software-like development cycle by effectively reducing developers' manual efforts for bug localization. @InProceedings{ASPLOS22p946, author = {Jiacheng Ma and Gefei Zuo and Kevin Loughlin and Haoyang Zhang and Andrew Quinn and Baris Kasikci}, title = {Debugging in the Brave New World of Reconfigurable Hardware}, booktitle = {Proc.\ ASPLOS}, publisher = {ACM}, pages = {946--962}, doi = {10.1145/3503222.3507701}, year = {2022}, } Publisher's Version Published Artifact Artifacts Available Artifacts Functional Results Reproduced