ASE 2026
41st IEEE/ACM International Conference on Automated Software Engineering (ASE 2026)

Title Page


Article: ase26foreword-fm000-p doi:

Welcome from the Chairs


Article: ase26foreword-fm001-p doi:

ASE 2026 Organization


Article: ase26foreword-fm002-p doi:

Sponsors


Article: ase26foreword-fm003-p doi:

SpecFSM: Extracting and Repairing Finite State Machines from Protocol Specification Documents
Xiangdong Li, DaWei Huang, Jingjing Guan, and Hui Li
(Beijing University of Posts and Telecommunications, Beijing, China)
Finite state machines (FSMs) are a fundamental abstraction for modeling dynamic system behavior and are widely used in protocol analysis. However, automatically extracting FSMs from natural-language specifications remains challenging. Protocol documents are often lengthy and structurally complex, with state-transition descriptions scattered across the text. As a result, large language models (LLMs) operating under incomplete context are prone to generating spurious transitions and logically flawed FSMs. In this paper, we propose SpecFSM, a framework for the automatic extraction and repair of protocol FSMs from natural-language specifications. SpecFSM combines state completion, evidence tracing, and formal verification into a unified pipeline that infers missing states, constrains incorrect transitions, and repairs logical defects in the extracted models. Experiments on multiple protocol documents show that SpecFSM improves the average number of correctly extracted state transitions by 54.7% over existing methods, while producing FSMs with stronger logical validity and better interpretability.

Article Search Artifacts Available Article: ase26main-p13-p doi:10.1145/3832783.3834331

Trace-Driven Automated PoC Synthesis for On-Chain Attacks
Xing Su, Hao Wu, Hanzhong Liang, Yunlin Jiang, Yuxi Cheng, Yating Liu, Fengyuan Xu, and Sheng Zhong
(Nanjing University, Nanjing, China)
Blockchain systems are increasingly targeted by on-chain attacks that exploit vulnerabilities in smart contracts, resulting in billions of dollars in losses. Understanding and reproducing these attacks is critical for defense, yet currently relies on manually crafting proofs-of-concept (PoCs), which is labor-intensive, expertise-driven, and difficult to scale. In this work, we take the first step toward automating PoC synthesis directly from transaction traces. Our key insight is that a concrete execution trace provides not only behavioral evidence for recovering exploit logic, but also runtime semantics that enable precise code synthesis and verification. We design TracExp, a trace-driven framework that combines a dual decompiler, which fuses symbolic lifting with concrete runtime values to produce semantically enriched pseudocode, and a sketch-based synthesis and refinement mechanism that grounds LLM generation in execution context and ensures semantic correctness. Evaluated on 321 real-world DeFi attacks collected over 20 months, TracExp achieves 94.70% generation, 83.80% compilable, and 57.01% verifiable rates, at an average cost of ∼$0.1 per case. Moreover, we contributed 33 previously unavailable PoCs within 2 days to DeFiHackLabs, and all were accepted and merged, earning $900 in bounties, demonstrating both the practical effectiveness and real-world impact.

Article Search Article: ase26main-p24-p doi:10.1145/3832783.3834332

Context-Aware Trust Verification for Identity-Based Software Signing
Chinenye Okafor, James C. Davis, and Santiago Torres-Arias
(Purdue University, USA)
Modern software release infrastructure uses identity-based software signing to associate software artifacts with a known identity. For example, registries such as npm and Docker Hub rely on Sigstore’s identity-based software signing in their artifact provenance workflows. However, existing verification procedures only provide evidence of a signer’s identity, but not the conditions under which signing occurred. As a result, compromised credentials, identity providers, or signing tools can still produce signatures that pass verification. This gap prevents software engineering tools (e.g., package registries, dependency analysis tools, deployment systems, and CI/CD verification stages) from enforcing verifiable context-aware trust policies for software signing.
We present DiVerify, a framework for automated verification of software signing conditions. A DiVerify signature binds identity claims from multiple independent scope providers, signing-environment attestation from a TEE-isolated daemon, and the signing key into a single verifiable object. A machine-checkable policy language specifies the identity, authentication, and environment conditions required for accepting a signature. We formally analyze DiVerify and show that it is sound, prototype it across three deployment modes, and demonstrate overhead under 400ms, with storage overhead of less than 1.6% of average PyPI package size. Case study integrations with existing systems confirm DiVerify composes well.

Article Search Article: ase26main-p30-p doi:10.1145/3832783.3834333

A11YRepair: Bridging Web Accessibility Barriers via Knowledge-Enhanced Divide-and-Conquer Repair
Kai Huang, Ling Zhu, Jian Zhang, Xiaofei Xie, and Chunyang Chen
(TU Munich, Germany; Nanyang Technological University, Singapore; Singapore Management University, Singapore)
Web accessibility (A11Y), which ensures web content is perceivable and usable for users with disabilities, is a critical requirement for modern web applications. Yet existing tooling overwhelmingly focuses on detecting A11Y violations rather than repairing them. Automated program repair (APR) techniques appear promising for this setting, but our study shows that state-of-the-art APR systems perform poorly when applied to real-world A11Y violations. Unlike conventional sparse-bug scenarios, web A11Y issues often manifest as multiple structurally related violations per page, requiring coordinated edits across multiple files. Existing repair systems fail to manage this multi-fault scale, as they handle each bug individually without considering their relationships or incorporating domain rules such as the Web Content Accessibility Guidelines (WCAG).
We propose A11YRepair, an LLM-based framework designed specifically for web A11Y repair. A11YRepair introduces a divide-and-conquer workflow that first clusters violations requiring coordinated edits to reduce redundant localization, and then decomposes each cluster by root cause so the LLM can generate focused and consistent patches. The framework further incorporates WCAG-driven knowledge to strengthen domain awareness during both fault localization and patch synthesis. To support systematic evaluation, we construct A11YBench, a benchmark of 60 real-world web projects collected from GitHub. Experimental results show that A11YRepair achieves higher repair effectiveness and lower cost than state-of-the-art baselines, and ablation studies confirm the importance of its divide-and-conquer design and selective domain knowledge integration. Specifically, patches generated by A11YRepair have been merged into open-source projects from Google, Microsoft, Facebook, IBM, K8s, Docker, and Alibaba, demonstrating its practical value in real-world dev scenarios. This work demonstrates the promise of LLM-driven accessibility repair and highlights the importance of combining structured repair strategies with domain expertise to advance inclusive software engineering.

Article Search Artifacts Available Article: ase26main-p37-p doi:10.1145/3832783.3834334

GALA: Graph-Augmented LLM Agents for Root Cause Analysis and Incident Response in Microservices
Yifang Tian, Yaming Liu, Zichun Chong, Zihang Huang, Yiran Li, and Hans-Arno Jacobsen
(University of Toronto, Canada)
Microservice root cause analysis (RCA) requires correlating failures across heterogeneous telemetry within complex service dependency graphs. Existing methods often rely on a single telemetry modality; recent LLM-based approaches can suffer from unconstrained exploration and hallucination; and most systems stop at fault ranking without producing actionable incident response. We present GALA+, a graph-augmented LLM agentic framework centered on graph-guided investigation, which uses service dependencies to bound exploration and refine diagnosis through localized multi-modal evidence. For initial hypothesis generation, GALA+ combines complementary telemetry signals with STRIX, a novel trace- and graph-structure-aware scoring module. GALA+ then produces ranked diagnoses, incident summaries, and stratified action recommendations. We further introduce SURE-Score, a human-guided evaluation framework co-developed with industry SRE experts for assessing RCA-specific output quality beyond conventional text similarity metrics. On two microservice benchmarks, GALA+ consistently achieves the strongest overall results, surpassing the best LLM-based baseline by more than 25 percentage points in AC@1, while also receiving the highest ratings from both SURE-Score and independent human SRE evaluation.

Article Search Article: ase26main-p46-p doi:10.1145/3832783.3834335

IntOAgent: A Practical Framework for Automated Detection and Triggering of Integer Overflow Vulnerabilities
Yuelin Wang, Jiongchi Yu, Xiaofei Xie, Yaohui Sun, Tianyu Shi, Yanbang Sun, Qiang Hu, and Junjie Wang
(Tianjin University, China; Singapore Management University, Singapore)
Integer overflows remain a long-standing yet often underestimated class of vulnerabilities in C/C++ software, often leading to logic errors and severe downstream security consequences. Despite their simple arithmetic nature, accurately detecting integer overflows in real-world codebases remains challenging. Existing approaches suffer from poor scalability due to the large search space of arithmetic operations, and often fail to generate triggerable proof-of-concept (PoCs) because jointly satisfying statement reachability and overflow-triggering constraints is inherently difficult.
In this paper, we propose IntOAgent, a scalable agent-based framework for automated detection and triggering of integer overflow vulnerabilities. IntOAgent is built on the key insight that reachability and overflow triggering should be treated as distinct yet coordinated tasks. To improve scalability, IntOAgent employs a two-stage filtering strategy to prune provably safe operations and a boundary-aware path prioritization to focus on overflow-prone executions. To enable effective PoC generation, IntOAgent infers input structures and integrates overflow-guided range analysis to drive inputs toward boundary violations while preserving path feasibility.
We evaluate IntOAgent on established benchmarks Juliet c and five widely-used real-world projects (SQLite, Libxml2, Libplist, V8, and Libpng) against state-of-the-art directed fuzzing and LLM-based techniques. IntOAgent outperforms existing approaches in both effectiveness and efficiency, discovering all existing vulnerabilities and 117 previously unknown integer overflow vulnerabilities (68 assigned CVEs). These results demonstrate that IntOAgent enables scalable generation of triggerable PoCs, significantly improving the practical detection of integer overflow vulnerabilities.

Article Search Article: ase26main-p66-p doi:10.1145/3832783.3834336

LLM-Based Agents for Identifying Bug-Introducing Commits
Niklas Risse and Marcel Böhme
(MPI-SP, Germany; CISPA Helmholtz Center for Information Security, Germany)
Śliwerski, Zimmermann, and Zeller (SZZ) just won the 2026 ACM SIGSOFT Impact Award for asking:
When do changes induce fixes?
Their paper from 2005 served as the foundation for a wide array of approaches aimed at identifying bug-introducing changes (or commits) from fix commits in software repositories. But even after two decades of progress, the best-performing approach from 2025 yields a modest increase of 10 percentage points in F1-score on the most popular Linux kernel dataset.
In this paper, we uncover how and why LLM-based agents can substantially advance the state-of-the-art in identifying bug-introducing commits from fix commits. We propose a simple agentic workflow based on searching a set of candidate commits and find that it raises the F1-score from 0.64 to 0.81 on the most popular Linux kernel dataset, a bigger jump than between the original 2005 method (0.54) and the previous SOTA (0.64). We also uncover why agents are so successful: They derive short greppable patterns from the fix commit diff and message and use them to effectively search and find bug-introducing commits in large candidate sets. Finally, we also discuss how these insights might enable further progress in root cause understanding and repair.

Article Search Article: ase26main-p90-p doi:10.1145/3832783.3834337

ARMOR: A Robust Self-Supervised Framework for Root Cause Analysis in Microservices under Missing Modality
Wenzhuo Qian, Hailiang Zhao, Ziqi Wang, Zhipeng Gao, Jiayi Chen, Zhiwei Ling, and Shuiguang Deng
(Zhejiang University, China)
Automated incident management is critical for microservice reliability. While recent unified frameworks leverage multimodal data for joint optimization, they unrealistically assume perfect data completeness. In practice, network fluctuations and agent failures frequently cause missing modalities. Existing approaches relying on static placeholders introduce imputation noise that masks anomalies and degrades performance. To address this, we propose ARMOR, a robust self-supervised framework designed for missing modality scenarios. ARMOR features: (i) a modality-specific asymmetric encoder that isolates distribution disparities among metrics, logs, and traces; and (ii) a missing-aware gated fusion mechanism utilizing learnable placeholders and dynamic bias compensation to prevent cross-modal interference from incomplete inputs. By employing self-supervised auto-regression with mask-guided reconstruction, ARMOR jointly optimizes anomaly detection (AD), failure triage (FT), and root cause localization (RCL). AD and RCL require no fault labels, while FT relies solely on failure-type annotations for the downstream classifier. Extensive experiments demonstrate that ARMOR achieves state-of-the-art performance under complete data conditions and maintains robust diagnostic accuracy even with severe modality loss.

Article Search Article: ase26main-p102-p doi:10.1145/3832783.3834338

Beyond Text Matching: Towards Reference-Free Evaluation for Human-Oriented Binary Reverse Engineering
Xiuwei Shang, Li Hu, Xiao Jiang, Jieke Shi, Junda He, Zhou Yang, Shaoyin Cheng, Guoqiang Chen, Weiming Zhang, and David Lo
(University of Science and Technology of China, China; Singapore Management University, Singapore; University of Alberta, Canada)
Human-Oriented Binary Reverse Engineering (HOBRE) aims to transform decompiled pseudocode into a more human-friendly representation, thereby reducing the cognitive burden of reverse analysis and improving efficiency. However, reliably evaluating HOBRE outputs remains a fundamental challenge. On one hand, human evaluation is costly, time-consuming, as well as difficult to scale. On the other hand, existing automated metrics either rely on executable test cases and runtime environments, which are often unavailable for real-world binaries, or depend on high-quality source code references that are typically inaccessible and fail to capture semantically equivalent but lexically diverse outputs. Although the emerging LLM-as-a-Judge paradigm is naturally well-suited to HOBRE evaluation, its effectiveness has not yet been fully studied.
This paper presents the first systematic investigation of the LLM-as-a-Judge paradigm for HOBRE, covering three representative tasks: function name recovery, binary code summarization, and decompilation optimization. We introduce BinJudgeBench, the first expert-annotated, reference-free evaluation benchmark based on multi-dimensional human judgment. Our empirical study reveals that LLM-as-a-Judge achieves an average correlation of 63.20% with human judgment, significantly outperforming traditional automated metrics at 35.04%. By analyzing the impact of various judge configurations, including backbone LLMs, prompting strategies, and decoding temperatures, on both correlation and cost, we find that no “one-size-fits-all” configuration exists, as the optimal setup varies across tasks and individual samples. To address this, we propose BinJudge, which employs a lightweight routing mechanism to adaptively select the optimal judge configuration for each specific task and sample. BinJudge improves correlation with human experts by 4.5%–24.7% and reduces API cost to 0.06×–0.84× of that of static best configurations, providing a scalable, cost-effective, and high-fidelity automated evaluation scheme for HOBRE.

Article Search Article: ase26main-p130-p doi:10.1145/3832783.3834339

Source-Free Detection and Impact Analysis of Compiler Optimization Problems in Mobile Applications
Han Hu, Xiaoheng Xie, Bo Sun, Jian Gu, Gang Fan, and Li Li
(Independent Researcher, Hong Kong; Independent Researcher, China; Monash University, Australia; Beihang University, China)
Mobile apps frequently suffer from frame drops, overheating, and excessive power consumption. While developers optimize algorithms and debug code, a critical bottleneck often goes unnoticed: native libraries compiled with low optimization levels (O0/O1 instead of O2/O3). Because these libraries execute without functional errors, the resulting performance degradation remains hidden in production apps.
We present OptDetect, a source-free framework that detects compiler optimization problems directly from app binaries. OptDetect handles mixed optimization levels through binary disassembly, chunk-level classification, and weighted score aggregation, achieving 93.0% accuracy on controlled datasets and 81.9% on real-world datasets. Applying OptDetect to 21,972 native libraries from 830 top-ranked Google Play apps, we find that 30.5% of libraries use low optimization levels, affecting 91.7% of apps.
Through case studies on 12 production apps, fixing detected issues reduces CPU instructions by 10-63% (median: 20.5%) for commercial apps and 15-58% (median: 32%) for open-source apps. Performance complaints decrease in 5 of 6 commercial apps, and ratings increase in 5 of 6. Further investigation reveals that widely-used third-party libraries are themselves distributed at low optimization levels, with 49.7% of 1,073 libraries in a major repository exhibiting this problem. These findings show that compiler optimization problems are common, source-free detectable, and practically consequential in mobile app ecosystems.

Article Search Article: ase26main-p135-p doi:10.1145/3832783.3834340

Quantum Squeeziness: An Information Theoretical Metric for Quantum Software Testability
Avner Bensoussan, Hector Menendez, and Mohammad Reza Mousavi
(King's College London, UK)
Testability—the degree to which faults can be revealed and diagnosed—has long been fundamental in software engineering but remains largely unexplored for quantum software. This work presents the first formalisation of testability for quantum programs, adapting classical concepts to quantum computation. We introduce Quantum Squeeziness, an information-theoretic metric inspired by classical squeeziness, to quantify how faults are masked and prevented from propagating to observable outcomes. We treat it as an inverse measure of testability: higher squeeziness implies stronger fault masking and reduced detectability.
We propose an efficient method to compute Quantum Squeeziness and validate it through large-scale quantum mutation analysis. Across more than 50k mutants, we observe a strong negative correlation between squeeziness and mutation score, showing that higher squeeziness predicts lower testability. We further introduce a technique that constrains where and how perturbations are injected, reducing cost while preserving accuracy. The computation is 15% faster than a standard error-propagation baseline, while the optimised variant, FastSqueeziness, reduces mean runtime from 3200s to 189s (16.9 times speedup) with 95% accuracy.
By establishing a rigorous foundation of testability for quantum software, this work opens new directions at the intersection of software engineering, quantum information theory, and testing. Beyond testing and debugging, it also supports applications in quantum error mitigation and correction, enabling principled assessment and improvement of program robustness.

Article Search Artifacts Available Article: ase26main-p151-p doi:10.1145/3832783.3834341

Spec2V: Planning-Guided Specification-to-Verilog Generation Framework with Verification-Driven Debugging
Xuming Liu, Chi Zhang, Jinglong Xu, Shan Zhou, and Jinbo Wang
(Technology and Engineering Center for Space Utilization at Chinese Academy of Sciences, China; University of Chinese Academy of Sciences, China)
Field Programmable Gate Array (FPGA) is essential in mission-critical fields like aerospace. It offers high flexibility and deterministic real-time response. With the increase in development demands and system complexity, the traditional design paradigm that relies on manual expertise has become inefficient. Although large language models (LLMs) provide a potential pathway for automated code generation, a significant semantic gap remains between the linear reasoning patterns of LLMs and the inherent parallelism of FPGA hardware logic. Current methods encounter difficulties in decomposing complex tasks, low-resource code generation, and resolving simulation errors.
To address these issues, we present Spec2V, an agent framework for the generation of Verilog code. Spec2V incorporates two key complementary components: (i) we propose a task planning method based on a dual-layer Tree of Thought. It decouples complex specifications into macro-topology and micro-logic structures to mitigate logical drift from linear reasoning and to improve the accuracy of initial solutions. (ii) During validation and debugging, Spec2V reformulates failure analysis as a structured attribution task with an LLM-as-a-Judge paradigm. It utilizes the reasoning capabilities of LLMs and multi-source evidence to localize the sources of simulation errors. Furthermore, the framework introduces a multi-temperature hedged sampling strategy and a waveform mismatch rate filtering mechanism to identify correct implementations efficiently from extensive candidate spaces. On the public VerilogEval Machine and Human benchmarks, Spec2V achieves Pass@1 accuracies of up to 99.8% and 99.4%, respectively. Case studies on representative RTL design tasks further demonstrate that Spec2V significantly reduces the manual effort required for implementation and mitigates the overhead of iterative engineering through closed-loop planning, validation, and debugging.

Article Search Artifacts Available Article: ase26main-p157-p doi:10.1145/3832783.3834342

Characterizing the Landscape of Open-Source Satellite Software
Jinfeng Wen, Qi Liang, Yuehan Sun, Federica Sarro, Ao Zhou, Xuanzhe Liu, and Shangguang Wang
(Beijing University of Posts and Telecommunications, China; University College London, UK; Peking University, China)
Satellites have become fundamental components of modern technological systems, supporting critical infrastructure in communication, navigation, Earth observation, and scientific research. As space exploration advances and demand for satellite-enabled services grows, reliance on complex, heterogeneous satellite software continues to increase. A systematic understanding of the satellite software landscape is therefore increasingly important, yet existing studies still lack a comprehensive empirical examination. To address this gap, we present the first characterization study of open-source satellite software, examining its ecosystem and development practices. We mine and analyze 22,286 satellite-related GitHub projects through three research questions on popularity trends (RQ1), software goals (RQ2), and development practices (RQ3). First, we characterize the temporal evolution of projects and active developers, revealing increasing popularity. Second, through manual inspection of 646 projects, we construct a taxonomy of 43 software-goal categories spanning conceptual design, datasets, system implementation, simulation, testing, and tools. Third, we conduct an in-depth analysis of projects with source code, revealing a highly heterogeneous and task-specialized ecosystem with 66 programming languages and diverse implementation strategies. Finally, we summarize key findings and derive actionable implications for satellite developers and researchers.

Article Search Article: ase26main-p161-p doi:10.1145/3832783.3834343

Bridging the Gap between Intent and Impact: An Empirical Study of GPU Optimizations in Deep Learning Frameworks
Siqi Wang, Fanjing Luo, Xing Hu, Xinyu Wang, and Xin Xia
(Zhejiang University, China)
Performance optimization is a critical concern in the evolution of deep learning frameworks, as GPU execution dominates training and inference costs. While hundreds of optimizations are proposed to accelerate training and inference, it is unclear how often these changes deliver meaningful gains in practice, and why they sometimes fail. To bridge this gap, we conduct an empirical study of GPU performance optimizations in deep learning frameworks. We analyze 202 performance-related pull requests from the major deep learning frameworks, and categorize them into 22 distinct optimization strategies (e.g., kernel fusion, auto-tuning, and vectorization). This taxonomy reveals a rich set of developer-intended performance improvements spanning logic, execution, and infrastructure levels.
Our evaluation reveals that optimization benefits are far from guaranteed. While some strategies achieve substantial speedups, a significant portion exhibit limited gains or even performance regressions when deployed in realistic workloads. To understand these discrepancies, we conduct multi-level benchmarking and microarchitectural profiling, uncovering three recurring root causes of performance degradation: (1) increased register pressure from aggressive fusion, (2) synchronization overhead migration in small kernels, and (3) memory arbitration under high-load kernels. These findings highlight a fundamental challenge in performance engineering: optimizations interact with hardware resources in complex and often counterintuitive ways, making their outcomes difficult to predict. Our study provides the first systematic characterization of these effects, offering insights to guide more reliable and performance-aware optimization in deep learning frameworks.

Article Search Article: ase26main-p162-p doi:10.1145/3832783.3834344

CIPIHunter: Detecting Configuration-Induced Prediction Instability in Deep Learning Frameworks
Yanzhou Mu, Shuo Meng, Mijung Kim, Xiang Chen, Chunrong Fang, Zhenyu Chen, and Juan Zhai
(UNIST, Republic of Korea; Nanjing University, China; Nantong University, China; University of Massachusetts at Amherst, USA)
Deep learning (DL) frameworks support diverse execution configurations, including precision modes and model optimizations. Even with a fixed model, input, and framework version, changing these configurations can change model predictions, causing Configuration-Induced Prediction Instability (CIPI). CIPI arises when configuration-dependent numerical perturbations propagate and cause potentially harmful model-level behavior. Existing API-level and model-level testing approaches are not well suited to detecting CIPI, as these approaches lack configuration-conditioned exploration and propagation-aware diagnosis. Therefore, we present CIPIHunter, which combines controlled configuration-pair testing, sensitivity-oriented mutation, feedback-guided search, and intermediate tracing to localize suspicious operators or operator groups. On 13 PyTorch models under a 100-round budget, CIPIHunter reports the highest raw CIPI count and 22 deduplicated Type I manifestations, 83.3% more than the strongest baseline, while remaining competitive in overall deduplicated manifestation coverage. In total, we submit 29 PyTorch issue reports. Developers have acknowledged eight, rejected five, and marked four as duplicates; nine remain pending, and three have been closed without an explicit determination. Separately, three reports are labeled “high-priority”. These results demonstrate the practical utility of CIPI testing for framework quality assurance.

Article Search Article: ase26main-p163-p doi:10.1145/3832783.3834345

TasmScan: Continuation-Aware Taint Analysis for TVM Bytecode with Savelist Abstraction
Yixuan Liu, Yin Wu, and Yi Li
(Nanyang Technological University, Singapore; Xi'an Jiaotong University, China)
The Open Network, with a peak market capitalization exceeding twenty billion US dollars and more than one hundred seventy five million activated on chain addresses, relies on the TON Virtual Machine to execute smart contracts. The TON Virtual Machine uses first class continuations with savelists to manage control flow and register state across continuation invocations. Because registers captured by savelists allow data to flow across continuation boundaries without passing through the operand stack, bytecode level analyses cannot construct complete data flow tracking without explicitly modeling savelist semantics. We present TasmScan, the first bytecode level static analysis framework for the TON Virtual Machine that enables cross continuation data flow reasoning without requiring source code. TasmScan models savelist semantics through forward register analysis with a formal over approximation guarantee for exactly resolved save sites and locally tracked register definitions. It then lifts bytecode into TASIR, a typed intermediate representation, and performs path sensitive taint analysis with context aware sources to detect defects. We evaluate TasmScan on two thousand nine hundred twenty one contracts from the TON verifier registry and a labeled benchmark of two hundred eight contracts with human confirmed ground truth. On the full corpus, TasmScan resolves two hundred ninety four thousand five hundred forty six dynamic continuation targets with one hundred percent precision. Ablation confirms that savelist propagation is essential for resolving indirect register calls that depend on cross continuation register passing. On the benchmark, TasmScan detects ninety five point three percent of defects across five classes with ninety six point eight percent precision. A stratified sample of three hundred sixty six pairs from the full corpus estimates eighty five point eight percent overall precision. TasmScan offers a seventeen times median speedup over the state of the art symbolic execution baseline and completes one hundred percent of analyses in the path analysis comparison with zero crashes or timeouts.

Article Search Article: ase26main-p182-p doi:10.1145/3832783.3834346

Quick Bug Detection through Black-Box Checking: A Systematic Evaluation
Bram Pellen, María Belén Rodríguez, Frits Vaandrager, and Petra van den Bos
(Radboud University, Netherlands; University of Twente, Netherlands)
Combinations of active automata learning, model-based testing and model checking have been successfully used in numerous applications, e.g., for spotting bugs in implementations of major network protocols and to support refactoring of embedded controllers. However, in the large majority of these applications, model checking is only used at the very end, when no counterexample can be found anymore for the latest hypothesis model. This contrasts with the original proposal of black-box checking (BBC) by Peled, Vardi & Yannakakis, which applies model checking for all hypotheses, also the intermediate ones. In this article, we present the first systematic evaluation of the ability of BBC to find bugs quickly, based on 77 benchmark models from real protocol implementations and controllers for which specifications of safety properties are available. Our main finding are: (a) In cases where the full model can be learned, BBC detects violations of the specifications with just 3% of the queries needed by an approach in which model checking is only used for the full model. (b) Even when the full model cannot be learned, BBC is still able to detect many violations of the specification. In particular, BBC manages to detect 96% of the safety property violations in the challenging RERS 2019 industrial LTL benchmarks. (c) Our results also confirm that BBC is way more effective than existing model-based testing algorithms in finding deep bugs in implementations.

Article Search Artifacts Available Article: ase26main-p302-p doi:10.1145/3832783.3834347

A Scalable Rule-Based Deep Reinforcement Learning Framework for the Next Release Problem
Hemanth Gudaparthi, Nan Niu, Prem Swaroopaanda Ramalingam, and Laxman Chowdary Kakara
(Governors State University, USA; University of North Florida, USA)
Next-release planning and feature selection remain critical challenges in the software deployment cycle, especially with the rapid introduction of updates and AI-driven functionalities. Selecting a subset of high-impact features that balances customer satisfaction with stakeholder business goals requires scalable models capable of producing near real-time predictions from requirements descriptions. However, uncertainty about which features truly align with customer needs makes this task difficult. This paper proposes a scalable feature prioritization framework that leverages rules derived from feature perplexity, probabilistic relevance, and customer-oriented criteria. We introduce a dual deep-reinforcement learning (DDRL) architecture that recommends high-value requirements through a hierarchical strategy. The first level applies graph reinforcement learning (GRL) guided by rule-based reasoning to structure tasks and reduce search space, while the second level uses deep reinforcement learning (DRL) to refine predictions. Experimental results on Zoom, Webex, and Teams datasets show that the approach maintains linear complexity (O(m)) compared to the cubic complexity (O(m^3)) of traditional graph-based methods, where m is the number of edges. The model achieves up to 85.56

Article Search Artifacts Available Article: ase26main-p350-p doi:10.1145/3832783.3834348

DynCA: An Effective Algorithm for 3-Wise Combinatorial Interaction Testing in Highly Configurable Systems
Ruizhi Shi, Shuangyu Lyu, Renyu Yang, Wei Wu, Chanjuan Liu, Chunming Hu, and Chuan Luo
(Beihang University, China; Central South University, China; Dalian University of Technology, China)
With the increasing demand for software customization, highly configurable systems have become increasingly important. Combinatorial interaction testing (CIT) is widely recognized as an effective technique for testing such systems. The most critical problem in CIT is constrained covering array generation (CCAG), which aims to construct a t-wise covering array (CA) with as small a size as possible, where t denotes the testing strength. Compared with the widely adopted pairwise testing ( 2-wise CIT), 3-wise CIT is capable of detecting more faults and provides greater practical benefits. However, it also introduces significantly higher computational complexity. Although many existing algorithms perform well on 2-wise CCAG, current state-of-the-art CCAG algorithms still suffer from the severe scalability challenge in 3-wise CCAG and have difficulty generating small-sized CAs for large-scale instances. Since the size of a CA directly affects testing efficiency in practice, generating small-sized 3-wise CAs is of great importance. In this work, we propose , a novel 3-wise CCAG algorithm that effectively generates small-sized CAs for large-scale instances. Experimental results on extensive real-world highly configurable systems demonstrate that, compared with existing state-of-the-art algorithms, reduces the size of generated 3-wise CAs by 10.8%. Moreover, is efficient, requiring only half of the running time to generate smaller 3-wise CAs than existing algorithms.

Article Search Artifacts Available Article: ase26main-p354-p doi:10.1145/3832783.3834349

Effective and Efficient Context Retrieval via Partial Dependency Graph for Repository-Level Code Generation
Zhongxin Liu, Zhonghao Jiang, Zhifan Ye, Haoye Wang, Jiakun Liu, and Xiaoxue Ren
(Zhejiang University, China; Hangzhou City University, China; Harbin Institute of Technology, China)
LLM-based repository-level code generation aims to generate code according to the available context within a software repository, requiring LLMs to understand and reason over complex code dependencies. Due to limited context windows and insufficient understanding of repository-specific code, LLMs typically rely on retrieval-augmented generation (RAG) to incorporate relevant contextual code. Early RAG approaches primarily employ similarity-based retrieval, which often fails to retrieve code snippets that the target function actually depends on. Recent work introduces graph-based retrieval to model such dependencies, but typically relies on manually designed rules and static global graphs, leading to limited flexibility and high construction and maintenance costs.
In contrast, human developers typically collect helpful context by implicitly constructing a partial dependency graph and iteratively inspecting along it. Inspired by human behavior, we propose DyRetriever, an effective and efficient context retrieval method via partial dependency graphs. DyRetriever leverages an LLM to simulate how human developers collect helpful context by first selecting a set of entry-point functions and then performing multi-hop reasoning along the code dependency graph. During multi-hop reasoning, DyRetriever leverages the LLM’s semantic understanding to explicitly validate whether a function can help to generate the target function, thereby eliminating reliance on manually designed rules and enabling flexibility across application scenarios. Instead of statically constructing a global dependency graph, DyRetriever builds a partial graph on demand and discards it after use, reducing construction overhead and avoiding maintenance costs. We integrate DyRetriever with a similarity-based code retriever to build a new code generation approach, DyCoder, and evaluate it on two widely used repository-level code generation benchmarks, CoderEval and DevEval. Experimental results show that DyCoder achieves relative Pass@1 improvements of 25.63% and 59.73% on CoderEval and DevEval, respectively, compared with existing RAG-based methods, while being 7.4× faster than baselines based on static dependency graph construction.

Article Search Article: ase26main-p356-p doi:10.1145/3832783.3834350

When Ambiguity Meets Atypicality: Dual-Perspective Test Input Prioritization for DNNs
Haoran Li, Shihai Wang, Bin Liu, Jialuo Chen, Wenjing Zhu, Yu Liu, Tengfei Shi, and Shudi Guo
(Beihang University, China; Zhejiang University, China)
While Deep Neural Networks (DNNs) have achieved remarkable progress in cutting-edge domains, their inherent brittleness has become a growing concern. To ensure the reliability and safety of DNN-enabled software, DNN testing has emerged as an indispensable practice. Within this context, test input prioritization is essential for early fault detection and reducing labeling costs. However, it remains challenging to accurately identify failure-inducing inputs. Although decision ambiguity and distributional atypicality are two widely adopted perspectives for characterizing inter-class competition and intra-class typicality respectively, relying on either perspective in isolation inevitably introduces blind spots. In this paper, we propose DuFP (Dual perspective Feature space Prioritization), a KNN density-based test input prioritization approach for DNNs that jointly incorporates both inter-class and intra-class perspectives. The prioritization framework of DuFP is built upon class-conditional density estimation. Based on the estimation results, prediction correctness is characterized by an ambiguity score and an atypicality score, with the former reflecting decision ambiguity and the latter quantifying distributional atypicality. A hybrid uncertainty score is then constructed by integrating both scores to guide the final prioritization. We evaluate DuFP on prioritization and selection tasks across image and text datasets under clean, corrupted, and adversarial scenarios. Experimental results demonstrate that DuFP effectively and efficiently prioritizes fault-inducing inputs and outperforms state-of-the-art approaches.

Article Search Artifacts Available Article: ase26main-p367-p doi:10.1145/3832783.3834351

Not In My Git Yard: Catching Backdoors at Commit and Release Time
Dimitri Kokkonis, Michaël Marcozzi, and Stefano Zacchiroli
(Université Paris-Saclay, France; CEA List, France; Télécom Paris, France)
Code-level backdoors—stealthy code changes that grant hidden privileges via secret triggers—pose a persistent threat to open-source software. Known attempts to inject such backdoors into widely used projects through malicious commits, tampered release packages, or compromised third-party dependencies, were stopped only by luck and manual review. Existing Continuous Integration (CI) pipelines cannot detect these attacks, and downstream binary analysis tools require substantial manual effort. In this work, we present Lily, an automated approach that strengthens open-source development and release processes against backdoor injection. Lily integrates a backdoor detection mechanism into (1) CI pipelines to block malicious commits, and (2) release vetting workflows to prevent tampered releases or compromised dependencies from entering large ecosystems, such as Linux distributions. Lily offers two key contributions. First, it enhances CI-compatible fuzzing with the capability to detect triggers of suspicious behavior based on historical and current software executions. This enables fast, precise backdoor detection suitable for both CI and update validation workflows. Second, it combines code change analysis with fuzzing data to precisely point maintainers to backdoor-revealing code regions, even when release updates modify millions of lines of code. We also outline five strategies attackers could use to evade Lily, and evaluate corresponding defenses. Our experiments across hundreds of benign and backdoored commits and releases show that Lily achieves high detection accuracy with low false alarm rates, reliably identifies malicious code, resists adversarial attempts, and would have prevented real-world backdoor incidents.

Article Search Article: ase26main-p379-p doi:10.1145/3832783.3834352

EffiHolmes: Differential Profiling-Guided Repository Level Time Inefficiency Fix Localization
Haowen Yang, Yun Peng, and Zishuo Ding
(Hong Kong University of Science and Technology, Guangzhou, China; Chinese University of Hong Kong, Hong Kong)
Large software systems often suffer from time inefficiencies, leading to excessive execution time despite functional correctness. Local- izing the fix locations of such inefficiency issues is notoriously difficult. Applying existing fault localization paradigms to this task presents significant challenges. Unlike functional bugs, time ineffi- ciencies do not cause crashes or test failures. They therefore provide neither binary oracles nor stack-trace localization clues, making traditional fault localization and recent LLM-based methods not ap- plicable. While runtime profiling provides alternative clues, it faces some challenges in repository-level settings. Single-run profiling is non-discriminative, failing to distinguish inefficiency hotspots (i.e., time-consuming functions) from execution noise. Furthermore, existing profilers struggle to extract the execution paths relevant to inefficiency from the vast background execution. Crucially, even with the extracted execution paths, a semantic gap still persists between the observed inefficiency and where to fix. In this paper, we propose EffiHolmes, an LLM-based frame- work for localizing fix locations of repository-level time inefficiency issues. First, it employs differential profiling (default vs. scaled workloads) to pinpoint inefficiency hotspots. Second, to extract the execution paths relevant to inefficiency, it performs execution path extraction, constructing compact execution paths that connect hotspots to the reported inefficient function. Third, to bridge the gap between the hotspots and fix locations, it employs guided LLM reasoning with domain heuristics to locate the inefficiency logic. To facilitate the evaluation, we introduce RepoEffi-Bench, the first benchmark for repository-level inefficiency localization, consisting of 140 high-quality time inefficiency issues collected from popular Python repositories. The results show that EffiHolmes consis- tently outperforms state-of-the-art retrieval-based, agent-based, and profiling-based baselines, with gains of 4.29 pp on GPT-5.1 file- level Acc@3 and 15.00 pp on qwen3-4b function-level Acc@5 over the best baseline. It also remains robust across model capacities.

Article Search Article: ase26main-p417-p doi:10.1145/3832783.3834353

Maximal Format-Free Data Repair
Zijian Luo, Xi Wu, Hong Jin Kang, Alan Fekete, and Rahul Gopinath
(University of Sydney, Australia)
ERROR in abstract.

Article Search Artifacts Available Article: ase26main-p428-p doi:10.1145/3832783.3834354

AgentExecutor: Partial Code Execution via Agentic Context Generation
Junkai Chen, Chengran Yang, Xing Hu, Zhenhao Li, Xin Xia, and David Lo
(Singapore Management University, Singapore; Zhejiang University, China; York University, Canada)
Executing code snippets is essential for dynamic program analysis, but it remains challenging to execute an arbitrary code snippet due to issues like missing context and incomplete dependencies. Existing approaches to partial code execution, such as LExecutor and Treefix, leverage the power of language models to infer missing information and enable execution. However, they suffer from (i) limited action spaces and feedback, and (ii) rigid optimization strategies, which restrict their effectiveness and efficiency. In this paper, we propose AgentExecutor, a novel multi-agent framework for partial code execution. Our approach introduces a three-phase design: execution environment preparation, dynamic exploration with iterative refinement, and prefix evolution via program synthesis. Supported by the power of LLM agents who can think, act, and get feedback iteratively, AgentExecutor is able to autonomously explore a richer action space, enabling diverse operations such as creating resource files and resolving environment configuration. Furthermore, it adopts adaptive optimization strategies, including coverage-guided context pruning and prefix evolution via program synthesis, to systematically improve the execution quality of partial code. We evaluate AgentExecutor on two widely used datasets comprising Stack Overflow snippets and open-source project code. The results show that AgentExecutor achieves up to 94% and 90% code coverage, outperforming the state-of-the-art approach Treefix by 19.9% and 13.8%, respectively. In addition, AgentExecutor significantly reduces execution time (by up to 80.3%) and cost (by up to 56.6%). These findings demonstrate that AgentExecutor provides an effective and efficient solution for partial code execution.

Article Search Article: ase26main-p477-p doi:10.1145/3832783.3834355

Automated Lemma Discovery in Agentic Program Verification
Huan Zhao, Haoxin Tu, Zhengyao Liu, Martin C. Rinard, and Abhik Roychoudhury
(National University of Singapore, Singapore; Massachusetts Institute of Technology, USA)
Deductive verification provides strong correctness guarantees for code by extracting verification conditions (VCs) and writing formal proofs for them. The expertise-intensive task of VC proving is the main bottleneck in this process, and has been partly automated owing to recent advances in Large Language Model (LLM) agents. However, existing proof agents are not able to discover helper lemmas – auxiliary lemmas that aid in proving – and thus fall short as programs grow in size and complexity.
In this paper, we argue that VC proving for program verification is more than a purely mathematical task, and benefits considerably from program comprehension. Our key insight is that human proof engineers often discover and apply helper lemmas based on their understanding of the program semantics, which are not directly reflected in the VCs produced by VC generators. Inspired by this insight, we propose an LLM agent, LemmaNet, that discovers helper lemmas in two ways. Specifically, the agent first synthesizes lemmas offline by directly analyzing the source code and specifications and then relating this semantic understanding to the mechanical, verbose encoding produced by VC generators. As the proof unfolds, LemmaNet then adapts existing helper lemmas online to accommodate evolving proof states, enabling the agent to effectively discharge complex VCs on-the-fly.
We implement LemmaNet on top of an existing proof agent AutoRocq for Rocq and the Frama-C ecosystem, and evaluate it on SV-COMP and established real-world subjects, including modules of the Linux kernel, Contiki OS, standard C++ library, and X.509 parser. Our experimental results demonstrate that LemmaNet significantly outperforms state-of-the-art approaches, highlighting the importance of program comprehension-aided lemma discovery in agentic program verification.

Article Search Article: ase26main-p529-p doi:10.1145/3832783.3834356

Regression Accumulation in Multi-turn LLM Programming Conversations
Yonghui (Andie) Huang, Lin Ma, Amjed Tahir, Qian Zhang, Liwen Xiao, and Lysa Xiao
(Massey University, New Zealand; Alibaba Research Center for Complexity Sciences, China; Hangzhou Normal University, China; University of Otago, New Zealand)
In LLM-assisted software development, coding is often iterative. In this paper, single-turn refers to settings where the LLM responds in one request with one code suggestion, whereas multi-turn refers to settings where the LLM responds across multiple sequential requests within the same conversation. Regression accumulation refers to a later code suggestion breaking requirements introduced in earlier turns. Reliability depends not only on satisfying the current turn, but also on preserving requirements in earlier turns. Yet, there are limited evaluations of the accumulation of regression in multi-turn coding conversations.
To address this gap, we conduct an empirical study of multi-turn regression in LLM programming. We construct 542 tasks from the HumanEval+ and MBPP+ benchmarks, and instantiate each task as an 8-turn requirement-evolution chain. We evaluate six LLM models on 26,016 turn instances (542×6×8). At each turn, we check if the current code suggestion still passes earlier benchmark tests, and we analyze 384 failure cases drawn from the failure population in our 26,016 turn instances to derive and validate a taxonomy of multi-turn regression bugs through independent four-annotator labeling. Our results show that regression accumulation appears across all six models, with 43% to 73% of tasks losing previously correct behavior over the full conversation. Final-turn quality is lower than initial-turn quality across models, and especially in turns adding input validation or broader input types. Manual analysis shows that Cross-Turn Conflict, where later code conflicts with earlier requirements, is the main failure class. We further find that Verification Gate, which checks new code against prior tests and triggers rollback and retry, is the only strategy that consistently improves all models, raising final-turn quality from 75.8% to 87.9% on DeepSeek-V3 and from 31.6% to 47.3% on Llama-3.1-8B.
These findings suggest that strong performance on single-turn tasks can overestimate reliability in multi-turn coding conversations. Future evaluation and tool design should test whether later code suggestions preserve requirements from earlier turns and should include Verification Gate mechanisms.

Article Search Artifacts Available Article: ase26main-p579-p doi:10.1145/3832783.3834357

TheBotCompany: Self-Organizing Multi-agent Systems for Continuous Software Development
Wenhan Lyu, Yue Xiao, Yixuan Zhang, and Yifan Sun
(William & Mary, USA)
Large language model (LLM)-based multi-agent systems have shown promise in automating software development tasks. However, most vibe-coding systems focus on completing small tasks and incremental code changes, leaving persistent, continuous software development largely unexplored. We present TheBotCompany, an open-source orchestration framework for continuous multi-agent software development. TheBotCompany introduces three key innovations: (1) a three-phase state machine (Strategy to Execution to Verification) for milestone-driven development, (2) self-organizing agent teams where manager agents dynamically hire, assign, and retire worker agents based on project needs, and (3) asynchronous human oversight. We evaluate TheBotCompany on real-world software projects over multiple days of continuous development, measuring team adaptation patterns, milestone completion rates, cost efficiency, and code quality. Our results demonstrate that the self-organizing approach enables effective long-term software development with measurable progress, while the verification phase catches defects that would otherwise persist.

Article Search Artifacts Available Article: ase26main-p597-p doi:10.1145/3832783.3834358

When Comments Mislead: An Empirical Study of Outdated Comments in LLM-Based Code Completion
Zhikun Xia, Yuan Huang, Xiangping Chen, and Zibin Zheng
(Sun Yat-sen University, China)
Large language models (LLMs) are widely used for code completion, where natural-language comments serve as key contextual signals. While prior work shows that comments and natural-language descriptions can improve generation quality by providing high-level semantic guidance, their reliability in evolving codebases remains unclear. In practice, comments may be outdated or inconsistent with the current implementation, raising the question of whether such context may mislead generation rather than simply degrade it.
In this paper, we present an empirical study of outdated comments in LLM-based code completion. We find that outdated comments tend to introduce a systematic misleading effect: instead of merely degrading performance, they can bias generated code toward historical implementations, producing outputs that remain globally plausible while failing in regions where the implementation has changed, as outdated comments often retain partially correct yet inconsistent information. Importantly, this effect is not uniform. Our study shows that the impact of outdated comments varies with factors such as the function signature length and task complexity, reflecting the interaction between structural signals and natural-language guidance. Overall, our findings challenge the common assumption that richer context is uniformly beneficial for code generation. They highlight that natural-language comments can act as strong but unreliable signals, and underscore the need for more robust handling of inconsistent contextual information in LLM-based code generation.

Article Search Artifacts Available Article: ase26main-p613-p doi:10.1145/3832783.3834359

Property-Based Testing of Wafer Handling Robots via Model-Based Planning and Fuzzing
Ruiyang Xu, Jingjing Liang, Guoyue Zheng, Geguang Pu, and Ting Su
(East China Normal University, China)
Wafer handling robots (WHRs) in semiconductor manufacturing require ultra-precise control software, where bugs can cause severe mechanical damage and costly downtime, making reliability validation critical. Yet no existing work addresses WHR validation.
In this experience paper, we introduce a property-based testing approach for WHRs. Our key observation is that the system requirements of WHRs could be used as the properties for bug detection.However, applying property-based testing to WHR systems is challenging due to the input generation challenge. Complex action dependencies make it difficult to generate valid test inputs, while enormous state spaces hinder achieving adequate test coverage. To address this, we propose model-based planning and fuzzing. Our insight is that the WHR can be represented as a state-based model. Based on this model, we systematically generate valid test inputs through backward planning, then explore diverse scenarios through parameter and environment mutation. We implemented our approach as the ArmGo tool and evaluated it on a production WHR deployed in industrial lithography equipment. ArmGo discovered 21 previously unknown bugs (all confirmed, 15 fixed), including axis limit violations, entanglement issues, and missing interlock checks. ArmGo has been deployed in the continuous integration (CI) pipeline of our industrial partner and is actively used for validating WHRs. We also have distilled a number of practical lessons learned based on our investigation.

Article Search Article: ase26main-p621-p doi:10.1145/3832783.3834360

ATLAS: Agentic Taxonomy of LArge-Scale Software Ecosystems
Junyi Lu, Mengyao Lyu, Jiahui Wu, Lei Yu, Chengwei Liu, Fengjun Zhang, Li Yang, Chun Zuo, and Yang Liu
(Institute of Software at Chinese Academy of Sciences, China; University of Chinese Academy of Sciences, China; Nanyang Technological University, Singapore; Nankai University, China; Sinosoft, China)
The open-source ecosystem on GitHub lacks a systematic hierarchical taxonomy of software repositories. GitHub Topics, the dominant organizational mechanism, is flat, inconsistent, and covers only 67% of repositories. We present ATLAS, the first framework that automatically constructs a hierarchical taxonomy for software repositories and classifies repositories into it end-to-end. By combining LLM global knowledge with real repository distributions, ATLAS proposes meaningful splitting dimensions and iteratively corrects those that fail to accommodate real repositories. A Designer Agent proposes splitting dimensions while a Classifier Agent assigns repositories; a self-corrective refinement loop uses classification failures to drive dimension revision through escalating strategies. We evaluate ATLAS on 54,387 GitHub repositories against six baselines spanning four paradigm families, two downstream software engineering tasks, and three model families. On a stratified 2,001-repository benchmark, ATLAS achieves a Taxonomy Quality F-score (TQF) of 83.13%, outperforming the best baseline by 15 percentage points (on the full 54k corpus the approximate TQF is 73.0%, a gap driven by Path Granularity's all-or-nothing scoring on longer paths rather than lower classification accuracy; see Section 4.5). It is the only method to simultaneously achieve high structural quality and high practical applicability. On downstream tasks, the ATLAS taxonomy enables software alternative discovery with P@1 = 85.71%, surpassing even human-curated lists (62.34%), and achieves the highest P@1 for repository retrieval. The taxonomy further reveals structural ecosystem trends that are difficult to obtain from flat tags or similarity methods: the shift from libraries to AI/ML applications (now 61% of newly community-adopted projects) becomes visible only through hierarchical, type-based categorization. An interactive taxonomy explorer is available at https://atlas-taxonomy.netlify.app/.

Article Search Artifacts Available Article: ase26main-p622-p doi:10.1145/3832783.3834361

SimServing: Native-Execution Simulation for Evolution-Resilient LLM Serving Configuration Tuning
Yujie Huang, Jiazhen Gu, Zhihan Jiang, and Michael R. Lyu
(Chinese University of Hong Kong, Hong Kong)
Large Language Model (LLM) services expose a large configuration space that critically affects performance, yet tuning these configurations on hardware (e.g., NVIDIA GPUs) is prohibitively expensive. Simulation offers a scalable alternative, but state-of-the-art simulators suffer from a fundamental software engineering pitfall: they re-implement the serving framework's complex control logic (e.g., request scheduling and memory management), creating a semantic copy that inevitably diverges from the rapidly evolving upstream software. Our empirical study across eight consecutive vLLM versions confirms this fidelity drift, with prediction errors widening as the framework evolves. We observe that the serving management software dominates performance fidelity yet accounts for less than 10% of total execution time, with GPU computation consuming over 90%. This asymmetry motivates SimServing, a hybrid simulation framework that executes the native, unmodified serving software while replacing GPU computation with a pluggable latency estimator. This design preserves the serving management software's control flow by executing it rather than modeling it, thereby automatically inheriting upstream updates and substantially reducing the maintenance burden associated with re-implementation. Integrating SimServing with Particle Swarm Optimization for configuration tuning, our evaluation shows that it reduces throughput prediction error from 13.8% to 4.9%, achieves up to 21.18× simulation speedup, and discovers configurations that improve throughput by up to 10.2% over existing baselines.

Article Search Article: ase26main-p625-p doi:10.1145/3832783.3834362

LLM Agents Can See Code Repositories
Dongjian Ma, Silin Chen, Yufei Yang, Yuling Shi, Yanfu Yan, and Xiaodong Gu
(Shanghai Jiao Tong University, China; Xi'an Jiaotong University, China; Zhejiang University, China)
Coding agents powered by large language models (LLMs) have demonstrated remarkable proficiency in software engineering tasks. Yet modern coding agents rely almost entirely on text, leaving a major gap between how human developers and agents comprehend software repositories. Human developers leverage visual repository cues such as folder hierarchies and dependencies, raising the question of whether multimodal models can similarly improve repository understanding. In this paper, we conduct the first systematic empirical study on multimodal foundation models for repository-level tasks. Our experiments across four modern multimodal models reveal that while a vision-only context representation degrades performance and inflates token costs, integrating visualized context graphs as a supplementary modality can help agents grasp the repository more efficiently. Specifically, providing agents with visual structural context alongside standard text interfaces reduces input token consumption by up to 26% while maintaining or improving issue-resolution accuracy. Furthermore, we demonstrate that visual tools are most effective when utilized during the fault localization stage and when agents autonomously dictate their exploration depth. Our findings highlight a promising hybrid-modality pathway for the design of next-generation coding agents.

Article Search Article: ase26main-p734-p doi:10.1145/3832783.3834363

Are Reasoning Traces Faithful in LLM-Based Code Vulnerability Detection?
Jialai Wang, Juncheng Lu, Songtao Yang, Bingyi He, Zeyu Gao, Chao Zhang, and Shuai Wang
(National University of Singapore, Singapore; Tsinghua University, China; Zhongguancun Laboratory, China; Huazhong University of Science and Technology, China; Hong Kong University of Science and Technology, Hong Kong)
Reasoning traces generated by large language models (LLMs) provide interpretability by exposing the factors that purportedly support their predictions. Such interpretability is important because model decisions may rely on spurious factors, and it can help practitioners identify such reliance and respond appropriately. However, the usefulness of reasoning traces hinges on their faithfulness, namely whether they reflect the factors that the model actually relies on when producing its prediction. Unfaithful traces may instead serve as plausible narratives that do not correspond to the true decision process. Despite its importance, reasoning-trace faithfulness has not been systematically studied in LLM-based vulnerability detection. We present VulFaith, the first framework for evaluating reasoning-trace faithfulness in this domain. We formalize two complementary criteria: completeness, which requires that all decision-relevant factors sustaining an insecure prediction are exposed, and per-factor faithfulness, which requires that each cited factor is genuinely relied upon by the model. To operationalize these criteria, we design an automated repair-based evaluation scheme that removes cited insecurity factors through code repairs and re-queries the model to test whether the prediction remains sustained. We implement VulFaith and conduct experiments on both repository-level and function-level vulnerability datasets, evaluating ten state-of-the-art open-source models and four commercial LLMs. Our results show that unfaithfulness is widespread, with completeness unfaithfulness reaching up to 66.14% and per-factor unfaithfulness up to 94.00%. We further validate that the unfaithfulness cases identified by VulFaith can support data augmentation for supervised fine-tuning, reducing completeness unfaithfulness by up to 47.89% and per-factor unfaithfulness by up to 59.18%.

Article Search Article: ase26main-p776-p doi:10.1145/3832783.3834364

CtxFuzz: Context-Aware Directed Fuzzing via Runtime Fuzzer-Agent Cooperation
Jiho Kim, Dongkwan Kim, HyungSeok Han, Youngjoon Kim, Soyeon Park, Sangwoo Ji, Joshua Wang, Dae R. Jeong, and Taesoo Kim
(Georgia Institute of Technology, USA; Microsoft, USA; Independent Researcher, USA; Samsung Electronics, Republic of Korea; Seoul National University, Republic of Korea)
Automated tools produce vulnerability reports at unprecedented scale, but turning a report into a confirmed proof-of-vulnerability remains challenging. Directed grey-box fuzzers are widely used for validation but lack the ability to consume the rich context these reports carry, reducing each target to a bare location. In contrast, LLM-based approaches can consume this context but are often unable to identify the true triggering mechanism or to synthesize a working proof-of-vulnerability.
We present CtxFuzz, a context-aware directed fuzzer that enables an LLM agent and a coverage-guided fuzzer to iteratively exchange and refine bug-triggering context throughout the campaign. The fuzzer contributes execution context, such as what inputs were tried and how the program behaved, grounding the agent’s reasoning in concrete evidence. The agent contributes semantic context, including hypotheses about the triggering conditions, explanations of failed attempts, and targeted mutations to overcome them, guiding the fuzzer toward relevant regions of the input space. Failed attempts are not discarded but decomposed into mutation scripts that translate each identified blocker into a fuzzer-executable strategy. This iterative exchange enables CtxFuzz to trigger vulnerabilities that neither directed fuzzers nor LLM agents reach independently.
On 62 vulnerabilities across 16 open-source projects from the AIxCC Final competition (39 C, 23 Java), CtxFuzz exposes 61 of 62 targets, including 29 that no state-of-the-art directed fuzzer reaches, and exposes two zero-day vulnerabilities through a real-world vulnerability report validation pipeline.

Article Search Article: ase26main-p797-p doi:10.1145/3832783.3834365

SustainSR: Interpretable OSS Sustainability Prediction via Symbolic Regression
Yang Deng, Mustafa Misir, Jinfu Chen, and Jifeng Xuan
(Wuhan University, China; Duke Kunshan University, China)
Open-source software (OSS) has become a foundational component of modern software ecosystems. Early prediction of OSS sustainability is crucial for identifying potential risks and supporting timely intervention.Existing approaches formulate OSS sustainability prediction task as a binary classification problem using Deep Learning (DL) models.However, their black-box nature obscures the decision process and undermines the credibility of the predictions.To address this limitation, we propose SustainSR, an interpretable symbolic regression–based method for OSS sustainability prediction.SustainSR reformulates prediction task as a function-learning problem and uses Genetic Programming (GP) algorithm to learn explicit sustainability expressions.We further apply Taylor expansion to quantify feature importance and pairwise interactions.The quantified results facilitate early risk identification and actionable intervention guidance.We evaluate SustainSR on Apache and GitHub projects from three dimensions: fitting performance, predictive performance, and interpretability.Results show that SustainSR significantly outperforms most baselines, it improves AUC by 1%-4% and F1_avg by 3%-7% over the best-performing baseline.Case studies further demonstrate that SustainSR offers actionable sustainability insights to support risk identification, project selection, and community governance.Overall, SustainSR provides an effective and interpretable solution for OSS sustainability analysis, bridging the gap between predictive performance and interpretability.

Article Search Article: ase26main-p823-p doi:10.1145/3832783.3834366

Stop When It Matters: Detectability-Guided Microbenchmarking for Performance Regression Testing
Zongxiong Chen, Derui Zhu, Jinfu Chen, Kundi Yao, Alexander Pretschner, Weiyi Shang, Manfred Hauswirth, and Sonja Schimmler
(Fraunhofer FOKUS, Germany; TU Munich, Germany; Wuhan University, China; Ontario Tech University, Canada; fortiss, Germany; University of Waterloo, Canada)
Performance microbenchmarks are widely used for detecting performance regressions in continuous integration, yet existing execution policies remain computationally expensive. We present detectability-guided microbenchmarking, a principled framework that reformulates benchmark execution as a Minimum-Detectable-Effect (MDE) problem, achieving substantial speedups with formal statistical guarantees. Our approach introduces a two-threshold specification: a lower threshold, δ₀, for negligible slowdowns, and an upper threshold, δ₁, for meaningful regressions. We theoretically and empirically show that this specification enables bounded false-positive rates below δ₀ and guaranteed statistical power around δ₁, while dramatically reducing execution time. To realize this framework, we propose a two-phase procedure: Phase 1 adaptively controls execution via online sample-size re-estimation guided by minimum detectable effect, enabling early stop when sufficient statistical resolution is reached and futile abort when target detectability is unattainable within the execution budget; Phase 2 performs regression analysis under the same specification, yielding decisions with provable error bounds. Experiments on 1702 microbenchmarks from three large-scale open-source projects demonstrate up to 93% reduction in execution time while empirical false-positive rates and power closely match their theoretical targets. Our findings reveal that detectability-guided execution offers a theoretically grounded yet practical path toward efficient microbenchmark-based regression detection.

Article Search Artifacts Available Article: ase26main-p833-p doi:10.1145/3832783.3834367

Evolution-Aware Regression Test Prioritization of ML-Enabled Systems using Gradient-Based Behavior Vectors
Eunho Cho, Donghwan Shin, and In-Young Ko
(KAIST, Republic of Korea; University of Sheffield, UK)
The machine learning(ML) component of an ML-enabled system evolves through retraining, fine-tuning, and optimization, so previously valid test results may no longer hold. A single evolution step can worsen performance on some test cases while improving others, making regression test prioritization inherently directional. We present Gradient-based Behavior Vector-Parameter Delta(GBV-PD), the first approach to operationalize the behavior vector space for evolution-aware regression test prioritization. GBV-PD represents each test case as a gradient-based vector(GBV), a low-dimensional projection of its loss gradient under the original model. It then projects the observed parameter update of the evolved model onto the same PCA basis and uses the resulting alignment to estimate whether each test case's loss is likely to increase or decrease, without running the evolved model on test cases during prioritization. In an empirical study across classification and regression tasks, GBV-PD consistently outperformed non-directional baselines and remained competitive with a full-gradient reference, while offering better time and storage profiles for repeated updates via reusable GBV caching. These results show that behavior-space ideas can be operationalized into a practical and efficient mechanism for repeated-update regression-test prioritization of evolving ML components within ML-enabled systems.

Article Search Artifacts Available Article: ase26main-p872-p doi:10.1145/3832783.3834368

Multi-level Code Optimization via Mixture of Prompts
Yun Peng, Jun Wan, Jiakun Liu, Shuzheng Gao, David Lo, and Xiaoxue Ren
(Fudan University, China; Zhejiang University, China; Harbin Institute of Technology, China; Chinese University of Hong Kong, Hong Kong; Singapore Management University, Singapore; Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security, China)
Runtime efficiency is a critical factor that impacts both software quality and user satisfaction. There are many approaches proposed for code optimization to improve runtime efficiency. Traditional code optimization methods operate on intermediate representations (IRs) during compilation for static languages. They are effective but struggle to handle dynamic languages that do not require compilation. Recently, large language models (LLMs) have been leveraged to directly optimize source code in dynamic languages. However, these methods fail to identify suitable optimization targets and usually conduct incomprehensive single-level optimization.
To address these challenges, we propose Optimo, a multi-level LLM-based code optimization approach built on a novel Mixture-of-Prompts (MoP) architecture. In the MoP architecture, Optimo identifies time-critical code structures as performance bottlenecks via differential profiling. These structures are then routed to some optimization strategies, akin to expert models in MoE, each tailored to optimize specific code patterns. Unlike traditional approaches that focus only on statement-level optimizations, Optimo operates at four levels of abstraction, ranging from coarse-grained algorithmic improvements to fine-grained optimizations in API usage. We evaluate Optimo on two code efficiency benchmarks, COFFE and Effibench. Our results demonstrate that Optimo achieves an up to 57.48% opt%, i.e., the percentage of optimized programs that are correct and at least 10% faster than the original programs, and an up to 3.97x speedup when optimizing human-written code, and it consistently outperforms the best baseline by up to 96.51% in terms of opt%. Furthermore, Optimo achieves an up to 42.42% opt% and an up to 13.51x speedup when optimizing LLM-generated code.

Article Search Article: ase26main-p874-p doi:10.1145/3832783.3834369

Evaluating the Maintainability of ViewModel Tests using Multi-modal DSLs: An Industrial Mixed-Methods Study
Mario Fuksa, Sandro Speth, and Steffen Becker
(University of Stuttgart, Germany; TU Munich, Germany)
As in many modern software engineering domains, the quality of an application's Graphical User Interface (GUI) is fundamentally verified by automated testing. One established approach to mitigate testing challenges is to leverage architectural patterns, such as Model-View-ViewModel (MVVM), to decouple presentation logic and test it in isolation. However, developing these ViewModel tests in general-purpose languages (GPLs) often forces engineers to write excessive boilerplate code for test setup and complex assertion chains. While multi-modal domain-specific languages (DSLs) show promise in mitigating these GPL-related issues, empirical evidence of their impact on test maintainability in practice is lacking. To address this gap, we present an industrial mixed-methods study evaluating how abstracting GUI tests to the ViewModel level via multi-modal DSLs impacts maintenance overhead in practice, using the ViMoTest method as our representative realization. After establishing industrial applicability via a pre-study comprising action research and a case study, we conducted a task-based evaluation with industry experts, triangulated with semi-structured interviews and a questionnaire. Quantitative results indicate that the ViMoTest method shows a strong tendency to reduce test specification effort and size compared to a C++ engineering baseline. Furthermore, qualitative expert feedback indicates that visual assertions and robust refactoring capabilities reduce cognitive load during structural test evolution. Overall, this study provides empirical evidence that multi-modal DSLs positively impact the maintainability of ViewModel tests.

Article Search Article: ase26main-p892-p doi:10.1145/3832783.3834370

Sound and Efficient Statistical Model Checking for Probabilities and Bounded Rewards
Hao Bu, Lin Huang, Tao Wei, and Jingyi Wang
(Ant Group, China; Zhejiang University, China)
Statistical model checking estimates quantitative properties of stochastic systems, such as the probability that a given property holds and the expected rewards. Ideally, an estimation algorithm should be both sound (i.e. satisfying the given confidence bound) and efficient (i.e. using fewer simulations or producing narrower confidence intervals). However, many existing algorithms are either unsound (e.g. relying on asymptotic approximations like the central limit theorem) or inefficient (e.g. using conservative bounds like the Hoeffding bound). In this paper, we present sound and efficient algorithms for estimating probabilities and bounded expected rewards under three settings: point estimation with absolute error, point estimation with relative error, and interval estimation, yielding six problems in total. For estimating probabilities, we employ a hybrid strategy integrating two existing algorithms in the point estimation with absolute error setting. We adopt suitable existing algorithms in the other two settings. For estimating bounded expected rewards, the unknown distribution shape makes it challenging to design sound and efficient algorithms. To address this, we propose a lifting framework to lift the three algorithms for estimating probabilities to three new algorithms for estimating bounded expected rewards. We formally prove the soundness of the three lifted algorithms and demonstrate their efficiency through experiments.

Article Search Article: ase26main-p914-p doi:10.1145/3832783.3834371

Tracing the Invisible: Semantic Message Flow Discovery via Data Contracts in Real-World Distributed Systems
Youlong Chen, Jie Lu, Mingtao Huang, Chenghang Shi, Yongheng Huang, Haofeng Li, Dong Liu, Mengna Ma, Yong Liu, Qinfen Hao, and Lian Li
(Institute of Computing Technology at Chinese Academy of Sciences, China; University of Chinese Academy of Sciences, China; Zhongguancun Laboratory, China; Qi An Xin Technology Group, China)
Distributed systems rely on complex messaging middleware, posing significant challenges for static analysis tools. Existing approaches often require manually specifying communication patterns between message senders and handlers—a process that is tedious, error-prone, and ill-suited to capture the semantic complexity of real-world systems with diverse routing logic. This paper introduces Hermes, an automated framework that leverages Large Language Models (LLMs) to bridge the semantic gap in distributed tracing. Key to our approach is the insight that "message types" serve as semantic contracts. Hermes leverages these types to synthesize latent routing logic into high-level communication patterns, which subsequently guide the precise discovery of concrete message flows from sender APIs to handlers. Evaluation on 10 real-world systems demonstrates that Hermes significantly outperforms existing approaches, achieving 88.6% precision and 99.6% recall in flow discovery. Applying Hermes to security auditing revealed 121 verified taint paths spanning 57 distinct sinks. These findings led to 56 confirmed zero-day vulnerabilities, of which 53 have been fixed and 7 assigned CVE IDs.

Article Search Article: ase26main-p924-p doi:10.1145/3832783.3834372

How Your Credentials Are Leaked by LLM Agent Skills: An Empirical Study
Zhihao Chen, Ying Zhang, Yi Liu, Gelei Deng, Yuekang Li, Yanjun Zhang, Jianting Ning, Leo Yu Zhang, Lei Ma, and Zhiqiang Li
(Griffith University, Australia; Wake Forest University, USA; Nanyang Technological University, Singapore; University of New South Wales, Australia; Zhejiang Sci-Tech University, China; University of Tokyo, Japan; University of Alberta, Canada; Independent Researcher, China)
Large Language Model (LLM) agents increasingly rely on third-party skills that operate within privileged execution environments and routinely handle sensitive credentials, yet how these credentials are leaked remains largely unexplored. To fill this gap, we present the first large-scale empirical study on credential leakage in agent skills. From 170,226 artifacts on SkillsMP, the largest open-source skill marketplace, we sampled 17,022 skills via stratified random sampling and analyzed each through static secret extraction (regex and AST parsing), dynamic sandbox testing of flagged candidates seeded with mock credentials, and cross-referencing developer intent against runtime behavior. Our analysis identifies 520 affected skills containing 1,708 security issues, and yields a taxonomy of 10 leakage patterns. Three findings stand out. First, 76.3% of cases require jointly analyzing natural-language descriptions and programming logic, showing that credential exposure in skills is fundamentally cross-modal. Second, Information Exposure accounts for 73.5% of vulnerability issues because agent frameworks feed stdout into the LLM context window, turning routine debugging into a credential exposure vector. Third, 89.6% of leaked credentials are immediately exploitable, and the fork-based distribution model defeats remediation, as secrets removed from upstream repositories persist across 50+ independent forks. Following responsible disclosure, all malicious skills have been removed. We release our dataset, taxonomy, and detection pipeline to support future agent security research.

Article Search Article: ase26main-p928-p doi:10.1145/3832783.3834373

WebCQ: Cooperative Multi-agent Deep Reinforcement Learning for Scalable Web GUI Testing
Yujia Fan, Sinan Wang, Zebang Fei, Yao Qin, Huaxuan Li, and Yepang Liu
(Southern University of Science and Technology, China)
Multi-agent reinforcement learning (MARL)-based techniques have shown promise for GUI testing. However, as the complexity of modern GUI software increases, existing MARL-based approaches (e.g., MARG and Fastbot) struggle to scale due to the inherent limitations of their underlying tabular reinforcement learning algorithms. This limits their applicability to large-scale commercial GUI software, especially web applications with vast state spaces and many interactive elements. To fill this gap, we propose WebCQ, a novel MARL-based approach for scalable web GUI testing. WebCQ incorporates QTRAN for multi-agent coordination and a lightweight synchronization mechanism, allowing it to work under asynchronous web testing scenarios. It extracts semantic and exploration features for each UI event to form an action vector. This vector is concatenated with the current state vector and fed into the policy network, enabling DQN-based decision making within a dynamic action space. We evaluated WebCQ on eight large-scale commercial websites. Under the same time budget and agent count, WebCQ explored 33.3% more states and executed 42.2% more unique actions than MARG, while triggering more failures on six of the eight websites under test. It also demonstrated strong scalability, maintaining higher action throughput during 20-hour experiments, and achieving greater performance improvements as the number of agents increased. These results show that WebCQ overcomes key limitations of existing MARL-based approaches, providing a scalable and effective solution to enhance modern web GUI testing.

Article Search Article: ase26main-p944-p doi:10.1145/3832783.3834374

MalSkills: Detecting Malicious Skills in the Agentic Supply Chain via Neuro-symbolic Reasoning
Shenao Wang, Junjie He, Yanjie Zhao, Yayi Wang, Kan Yu, and Haoyu Wang
(Huazhong University of Science and Technology, China; Ant Group, China)
Skills are increasingly used to extend LLM agents by packaging prompts, code, and configurations into reusable modules. As public registries and marketplaces expand, they form an emerging agentic supply chain, but also introduce a new attack surface for malicious skills. Detecting malicious skills is challenging because relevant evidence is often distributed across heterogeneous artifacts and must be reasoned in context. Existing static, LLM-based, and dynamic approaches each capture only part of this problem, making them insufficient for robust real-world detection. In this paper, we present MalSkills, a neuro-symbolic framework for malicious skills detection. MalSkills first extracts security-sensitive operations from heterogeneous artifacts through a combination of symbolic parsing and LLM-assisted semantic analysis. It then constructs the skill dependency graph that links artifacts, operations, operands, and value flows across the skill. On top of this graph, MalSkills performs neuro-symbolic reasoning to infer malicious patterns or previously unseen suspicious workflows. We evaluate MalSkills on a benchmark of 200 real-world skills against 5 state-of-the-art baselines. MalSkills achieves 93% F1, outperforming the baselines by 5-87 percentage points. We further apply MalSkills to analyze 150,108 skills collected from 7 public registries, flagging 620 potentially malicious skills. Through manual review, we confirm that 400 of them are indeed malicious, all of which were responsibly reported and are currently awaiting confirmation from the platforms and maintainers. These results demonstrate the practical potential of MalSkills in securing the agentic supply chain.

Article Search Article: ase26main-p951-p doi:10.1145/3832783.3834375

Less Is More: Tuning Configurable Systems with Imperfect Fidelity
Yulong Ye, Miqing Li, and Tao Chen
(University of Birmingham, UK)
Configuration tuning is essential for optimizing the performance of highly configurable systems, e.g., throughput or runtime, under a given environment. Yet, this is a challenging process as there can be many options to tune, and configuration measurement is often highly expensive. In this paper, we demonstrate the phenomenon of “less can be more”: system configuration tuning can be greatly improved with much superior budget utilization by partially tuning under the imperfect-fidelity—an environment that is similar, but cheaper to measure, compared with the concerned perfect-fidelity of environment under which the system should be tuned. We codify a conceptual framework of fidelity for configurable systems, drawing on which allows us to propose MFTune, a tuner that proactively explores in the space of >10⁴ possible imperfect-fidelity settings to approximate a useful one, which strikes for the wideness of tuning. This creates high-quality seeds for the perfect-fidelity, which in turn ensures the tuning depth. Experiment results against 10 state-of-the-art tuners, obtained from running diverse real-world systems for 19 months 24 × 7, show that MFTune performs considerably better on 83.33% cases with up to 19.34% improvement while achieving hours of budget saving in general.

Article Search Article: ase26main-p988-p doi:10.1145/3832783.3834376

Programming by Chat: A Large-Scale Behavioral Analysis of 11,579 Real-World AI-Assisted IDE Sessions
Ningzhi Tang, Chaoran Chen, Zihan Fang, Gelei Xu, Maria Dhakal, Yiyu Shi, Collin McMillan, Yu Huang, and Toby Jia-Jun Li
(University of Notre Dame, USA; Vanderbilt University, USA)
IDE-integrated AI coding assistants, which operate conversationally within developers' working codebases with access to project context and multi-file editing, are rapidly reshaping software development. However, empirical investigation of this shift remains limited: existing studies largely rely on small-scale, controlled settings or analyze general-purpose chatbots rather than codebase-aware IDE workflows. We present, to the best of our knowledge, the first large-scale study of real-world conversational programming in IDE-native settings, analyzing 74,998 developer messages from 11,579 chat sessions across 1,300 repositories and 899 developers using Cursor and GitHub Copilot. These chats were committed to public repositories as part of routine development, capturing in-the-wild behavior. Our findings reveal three shifts in how programming work is organized: conversational programming operates as progressive specification, with developers iteratively refining outputs rather than specifying complete tasks upfront; developers redistribute cognitive work to AI, delegating diagnosis, comprehension, and validation rather than engaging with code and outputs directly; and developers actively manage the collaboration, externalizing plans into persistent artifacts, and negotiating AI autonomy through context injection and behavioral constraints. These results provide foundational empirical insights into AI-assisted development and offer implications for the design of future programming environments.

Article Search Article: ase26main-p1071-p doi:10.1145/3832783.3834377

Understanding Bugs in Modern Agentic Frameworks: A Study of Symptoms, Root Causes, and Triggering Conditions
Xiaowen Zhang, Hannuo Zhang, and Shin Hwei Tan
(Concordia University, Canada)
Modern agentic frameworks such as CrewAI and AutoGen have evolved into complex, autonomous multi-agent systems, introducing reliability challenges that go beyond earlier pipeline-based LLM libraries. However, existing empirical studies focus on earlier LLM libraries or task-level bugs, leaving the unique complexities of these agentic frameworks unexplored. We present a comprehensive study of 409 fixed bugs across five representative agentic frameworks, proposing a five-layer architectural abstraction. Our taxonomy identifies previously unreported symptom categories—Unexpected Execution Sequence, User Configuration Ignored, and Incomplete/Incorrect Trace—and isolates agent-specific root causes including Model-Related Fault, Cognitive Context Mismanagement, and Orchestration Fault. Notably, the model integration layer is the most bug-prone yet receives disproportionately low test inclusion rate during bug fixing (47%), revealing a critical validation gap. Despite varying design paradigms, bug symptoms, root causes, and bug-prone components show substantial cross-framework consistency (JS similarity 0.62–0.88). Finally, we present the first systematic study of bug-triggering conditions, identifying error-prone factor combinations across element configurations, input patterns, and operations, and demonstrate their transferability across frameworks, providing a foundation for test oracle design and cross-framework benchmark.

Article Search Artifacts Available Article: ase26main-p1085-p doi:10.1145/3832783.3834378

Neuro-symbolic Requirements Elicitation: Utilizing Formal Verification Counterexamples as Contextual Prompts
Okba Tibermacine and Chouki Tibermacine
(National School of Artificial Intelligence, Sidi Abdallah, Algeria; University of Southern Brittany, France)
Incomplete and ambiguous requirements remain a common source of defects in safety-critical software systems. Formal methods provide rigorous correctness guarantees but rely on precise specifications, while Large Language Models (LLMs) enable flexible natural-language processing but lack logical consistency guarantees. Bridging the gap between informal stakeholder intent and formally verifiable specifications remains a key challenge in automated software engineering. This paper presents C-GRiD (Counterexample-Guided Requirements Discovery), a neuro-symbolic framework that integrates model checking into the requirements elicitation process. C-GRiD uses an LLM to synthesize an initial system model and a set of candidate requirements properties, which are then analyzed by a symbolic verifier. Counterexamples generated by the verifier are converted into structured natural-language queries that elicit missing constraints from stakeholders. This enables an iterative refinement loop that progressively reduces specification incompleteness while maintaining logical grounding of generated artifacts. We evaluate C-GRiD on 10 benchmark specifications using a controlled constraint-removal protocol to measure requirement recovery. Under a perfect oracle assumption, the framework achieves a mean constraint recovery recall of 79.2%. To assess robustness under realistic interaction settings, we introduce a simulated noisy stakeholder with a 25% error rate. In this setting, C-GRiD maintains a recall of 70.8%. Results show consistent improvements over one-shot and template-based baselines, demonstrating that counterexample-guided refinement can support the construction of more complete and logically grounded software specifications.

Article Search Article: ase26main-p1198-p doi:10.1145/3832783.3834379

SmartPoC: Validating Smart Contract Vulnerability Findings through PoC Generation and Execution
Longfei Chen, Ruibin Yan, Taiyu Wong, Yiyang Chen, Jialai Wang, and Chao Zhang
(Tsinghua University, China; Peking University, China; National University of Singapore, Singapore)
Smart contracts are commonly audited through static analysis to explore vulnerabilities. However, static approaches typically produce heterogeneous findings rather than reproducible, executable proof-of-concept (PoC) test cases, leading to costly and ad hoc manual validation. Large language models (LLMs) offer a promising way to translate audit reports into PoC test cases, but face three major challenges: noisy inputs, lack of execution grounding, and missing runtime oracles. We present SmartPoC, an end-to-end approach for validating reported vulnerabilities in audit reports by generating and executing PoC test cases with automated exploitability verification. SmartPoC first extracts a focused function-level slice from each report to reduce noise, centering on the key functions referenced in a finding and augmenting them with execution-relevant neighbors. To improve executability, we wrap LLM-based PoC synthesis in a generate–repair–execute loop, combining deterministic pre-execution sanitization with feedback-driven post-execution debugging. We further use differential verification as an oracle to confirm the exploitability of generated test cases. On the SmartBugs-Vul and FORGE-Vul benchmarks, SmartPoC achieves confirmation precision of 98.32% and 98.65%, with recall of 84.17% and 85.28%, respectively. On a recent Etherscan verified-source corpus, SmartPoC confirms 64 bugs from 545 audit findings at an average cost of $0.03.

Article Search Article: ase26main-p1210-p doi:10.1145/3832783.3834380

Every Scenario Matters: Scenario-Guided White-Box Testing for Static Code Checkers Powered by LLMs
Jun Liu, Liqiang Ji, Jiwei Yan, Linjie Pan, Jinsong Ren, Jun Yan, and Jian Zhang
(Institute of Software at Chinese Academy of Sciences, China; Huawei Technologies, China)
In static analysis tools, each detection rule corresponds to a dedicated code checker. The checker's quality directly determines the validity of reported defects. Thus, comprehensive testing of these checkers is crucial. Existing checker testing approaches usually generate test cases from high-level rule descriptions while treating checkers as black boxes. However, these descriptions usually omit implementation-level conditions and checking scenarios. To bridge this gap, our key insight is that the rule description and checker implementation provide complementary information for comprehensive testing. The implementation supplies concrete checking logic, while the rule anchors the intended semantics.
Building on this, we propose ScenSCT, a novel scenario-guided white-box checker testing approach powered by LLMs. ScenSCT first translates the checker's source code into a unified logic-based representation (CheckerIR) and applies both logic-decomposition and scenario-validation strategies to extract concrete checking scenarios. It then converts these scenarios into test cases through execution-feedback-guided test alignment to instantiate scenario constraints and scenario-invariant test augmentation to generate variants that preserve the target scenario. We evaluate ScenSCT on real-world Java checkers from five mainstream static analysis tools. In single-tool evaluation, ScenSCT extracts an average of ~9 scenarios and generates ~39 tests per checker, uncovering 270 potential issues (58 false positives and 212 false negatives) with an overall precision of 86.4%. Furthermore, cross-tool evaluation reveals 177 capability differences among shared checkers. Overall, ScenSCT offers a systematic testing method to improve the reliability of static analysis tools.

Article Search Article: ase26main-p1253-p doi:10.1145/3832783.3834381

Adaptive Proof Refinement with LLM-Guided Strategy Selection
Minghai Lu, Zhe Zhou, Danning Xie, Songlin Jia, Benjamin Delaware, and Tianyi Zhang
(Purdue University, USA; Meta, USA)
Formal verification via theorem proving enables rigorous proofs of software correctness, but it is difficult to scale due to the significant manual effort and expertise required. While Large Language Models (LLMs) have shown potential in automating proof generation, they frequently produce incorrect proofs on the first attempt that require iterative refinement to fix. However, most existing approaches employ fixed refinement strategies and cannot dynamically choose an effective strategy, which limits their performance. To overcome this limitation, we introduce Adapt, a novel proof refinement framework that leverages an LLM-guided decision-maker to dynamically select a suitable refinement strategy according to the state of the proof assistant and available context of an incorrect proof. We evaluate Adapt on two benchmark suites against five existing methods and find that it significantly outperforms the best baseline on both by proving 11.26% and 18.58% more theorems, respectively. Furthermore, we demonstrate Adapt’s generalizability by evaluating it across six different LLMs. We also conduct ablation studies to measure the contribution of each component and compare the trade-offs of alternative decision-maker designs.

Article Search Article: ase26main-p1291-p doi:10.1145/3832783.3834382

Assessing the Cross-Version Applicability of Java Library Vulnerability Exploits
Zirui Chen, Qi Zhan, Jiayuan Zhou, Xing Hu, Xin Xia, and Xiaohu Yang
(Zhejiang University, China; Queen's University, Canada; Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security, China)
Open-source software supply chain security relies heavily on assessing affected versions of library vulnerabilities. While prior studies have leveraged exploits for verifying vulnerability-affected versions, they point out a key limitation that exploits are version-specific and cannot be directly applied across library versions. Despite being widely acknowledged, this limitation has not been systematically validated at scale, leaving the actual applicability of exploits across versions unexplored. To fill this gap, we conduct the first large-scale empirical study on exploit applicability across library versions. We construct a comprehensive dataset consisting of 259 exploits spanning 128 Java libraries and 28,150 historical versions, covering 61 CWEs that account for 76.3% of vulnerabilities in Maven. Leveraging this dataset, we investigate the root causes of inconsistencies between exploit cross-version execution behaviors and ground truth, while also exploring strategies for exploit migration.
Our results (RQ1) show that, even without migration, exploits achieve 83.0% recall and 99.3% precision in identifying affected versions in Java, outperforming most widely used vulnerability databases and assessment tools. Notably, this capability enables us to contribute more than 1,400 confirmed missing affected versions to the CPE dictionary. We investigate the remaining exploit failures (RQ2) and find that they mainly stem from compatibility issues introduced by library evolution and changing environmental constraints. Based on these observations, we manually migrate exploits for 1,885 versions and distill a taxonomy of 10 strategies from these successful adaptation cases (RQ3), thereby increasing the overall recall to 96.1%. Our findings underscore the cross-version potential of exploits and lay a foundation for future research.

Article Search Article: ase26main-p1292-p doi:10.1145/3832783.3834383

TDRepro: A Neurosymbolic Approach to Reproducing Timing-Dependent Flaky Test Failures
Shanto Rahman, Talank Baral, August Shi, and Wing Lam
(University of Texas at Austin, USA; George Mason University, USA)
Tests that nondeterministically pass or fail on the same version of code are known as flaky tests, and their failures substantially hinder regression testing and debugging. Reproducing failures from timing-dependent (TD) flaky tests is particularly challenging because these failures depend on rare thread interleavings that may not occur even after running the test many times. Prior techniques reproduce such failures by searching for locations at which to inject delays during execution, but this search process can be expensive.
We present TDRepro, an automated approach that leverages large-language models (LLMs) and program analysis to efficiently reproduce TD test failures. TDRepro first identifies relevant methods by collecting executed methods during the test run and identifying those that execute concurrently with other methods. It then ranks these methods by computing the embedding similarity between each method’s code and the observed failure log. Finally, TDRepro uses an LLM to identify candidate locations for delay injection within the top-ranked methods. TDRepro validates these candidates by injecting delays and rerunning the test, iteratively refining the candidate locations based on unsuccessful attempts until the target failure is reproduced or the maximum number of iterations is reached. We evaluate TDRepro on 82 TD tests across 25 modules from open-source Java Maven projects. TDRepro reproduces 100% more failures than prior state-of-the-art technique, FlakeRake, and achieves more reliable reproduction, with 55 tests reaching a failure reproduction rate of >99%, compared with 30 tests for FlakeRake. Repeatedly reproducing failures with TDRepro requires 1482.11× and 36.5× less time and incurs 1104.2× and 26.9× lower cost than repeatedly rerunning tests and using FlakeRake, respectively.

Article Search Article: ase26main-p1337-p doi:10.1145/3832783.3834384

Think Before You Code: Dual Reasoning for the NLSafety–Utility Trade-Off in LLM Code Generation
Honghao Tan, Haibo Wang, and Shin Hwei Tan
(Concordia University, Canada)
Large language models (LLMs) for code generation are typically evaluated on functional correctness alone, overlooking whether generated code propagates harmful content embedded in the prompt. Prior work has shown that most Code LLMs reproduce offensive identifiers from injected renaming instructions without warning, yet existing approaches focus on detecting harmful content, neglecting functional correctness. Grounded in the Theory of Dual Channel Constraints (which states that code is a dual-channel medium combining an algorithmic (AL) channel for machine execution and a natural language (NL) channel for human communication, creating a unique safety-utility trade-off where a model must balance functional execution with responsible communication), we propose NLSafety-Utility Duality Score (SUDS), a metric that unifies code utility, safety adherence, and warning awareness into a single score across 12 ranked response scenarios, and Dual Reasoning (DR), a structured inference-time technique that requires an explicit safety audit and task-grounded code review before code generation. Evaluated on five LLMs across two benchmarks augmented with harmful keyword injections (820 and 2,135 samples), DR consistently achieves the highest SUDS across all models, improving mean SUDS by 1.32× to 3.42× over the baseline, while chain-of-thought prompting yields negligible safety gains and a safety-aware prompt provides only partial improvement. Further analysis reveals that DR's effectiveness scales with model capacity, that the one-shot exemplar primarily stabilizes output format for smaller models, and that structured reasoning cannot compensate for models with limited safety vocabularies.

Article Search Article: ase26main-p1364-p doi:10.1145/3832783.3834385

Fragmented Markers, Mixed Results: A Systematic Review of AI Coding Assistants and Developer Productivity
Annemarie Wittig, Alina Mailach, and Norbert Siegmund
(Leipzig University, Germany)
AI coding assistants are increasingly embedded in software engineering practice, yet the literature still offers no cumulative, easily interpretable picture of their effects on software developer productivity. The central problem is not a lack of studies, but a lack of comparability: prior work operationalizes productivity through heterogeneous markers, settings, and reporting styles, making cross-study conclusions difficult. To address this, we conduct a staged systematic literature review. In the first stage, we synthesize studies that explicitly examine developer productivity. In the second, we extend the scope to productivity-related developer outcomes using SPACE as a theory-informed lens, capturing human-centered evidence that explicit productivity studies often omit. Across both stages, we analyze how productivity is operationalized, which confounding factors are discussed, and how reported effects are distributed.
We identify 187 unique productivity and productivity-related markers, while individual studies typically only use five, revealing a broad but highly fragmented evidence base. Explicit productivity research is dominated by time-, output-, and artifact-oriented proxies, whereas the broader SPACE-aligned corpus makes satisfaction, psychological aspects, and well-being substantially more visible. Reported effects are often positive, but this pattern is stronger in perceived and tendency-based findings than in objective or statistically evaluated ones. Overall, the literature suggests that AI coding assistants can support productivity, but current evidence does not yet support simple, context-independent conclusions. The key contribution of this review is therefore a structured basis for making productivity evidence more interpretable, multidimensional, and comparable across studies.

Article Search Artifacts Available Article: ase26main-p1373-p doi:10.1145/3832783.3834386

How Reasoning Shapes Social Bias in LLM-Generated Code?
Weifeng Sun, Jieke Shi, Zhou Yang, Yuchen Chen, Hongyan Li, Meng Yan, and David Lo
(Singapore Management University, Singapore; University of Alberta, Canada; Alberta Machine Intelligence Institute, Canada; Nanjing University, China; Chongqing University, China)
Large language models (LLMs) are increasingly used for code generation, but recent studies have shown that generated programs can exhibit code bias, i.e., unfair or differential treatment encoded in code logic with respect to sensitive demographic attributes. Although prior work has investigated this problem in a direct generation setting, it remains unclear how bias behaves in reasoning-based code generation, where models produce intermediate reasoning traces before generating the final code. To bridge this gap, we conduct the first systematic empirical study of social bias in reasoning-based code generation. We evaluate 9 models spanning both standard LLMs and large reasoning models (LRMs) on realistic bias-sensitive coding tasks covering three human-centered decision scenarios. Our results reveal four main findings. First, reasoning generally reduces code bias, lowering the average bias rate from 0.64 to 0.40, but the improvement is highly model-dependent. Second, reasoning-based generation does not consistently preserve code quality, with the average quality dropping from 0.72 to 0.59. We further find that biased reasoning strongly predicts biased code, and that simply adjusting generation configurations is insufficient for robust bias mitigation. Motivated by these findings, we propose ProbeDebias, a reasoning-aware debiasing framework that detects biased reasoning traces and rewrites them before final code generation. Extensive experiments demonstrate that ProbeDebias achieves 87.76

Article Search Article: ase26main-p1394-p doi:10.1145/3832783.3834387

From Runnable Code to Shippable Applications: Test-Driven Development for Full-Stack Web Application Generation
Yuxuan Wan, Tingshuo Liang, Jiakai Xu, Jingyu Xiao, Yintong Huo, and Michael R. Lyu
(Chinese University of Hong Kong, Hong Kong; Columbia University in the City of New York, USA; Singapore Management University, Singapore)
Coding agents can generate runnable web applications, but their outputs frequently fail to satisfy functional requirements. Although test-driven development (TDD) offers a principled repair loop, applying it to web applications requires deriving executable acceptance tests, validating behavior through dynamic browser interactions, and translating observed failures into actionable feedback. We present TDDev, an experimental instrument that automates these tasks and enables closed-loop TDD with minimal human mediation. Using TDDev, we conduct the first controlled study of TDD for full-stack web application generation across 20 diverse web applications, multiple backbone models, two coding agents, and three TDD implementations. With sufficiently capable backbones, TDD improves accuracy by 15.5-23.7 percentage points and remains effective across both minimal and full-featured coding agents; replacing unreliable feedback with a stronger tester restores positive gains for a lower-capability backbone. Incremental TDD provides bounded control, Whole-Project TDD enables coordinated repair and achieves the highest cost efficiency in most configurations, and Agentic TDD performs best with highly capable models. Based on these findings, we provide a practical decision tree that selects an implementation according to model capability, feedback reliability, and deployment objectives. By automating acceptance testing and failure-guided repair, TDDev also reduces the human testing and intervention required during development.

Article Search Article: ase26main-p1397-p doi:10.1145/3832783.3834388

Planner-Fuzz: Overcoming Diagnostic Saturation in UAV Planners via Boundary-Oriented Fuzzing
Rongbo Chen, Chengcheng Zhao, Peng Cheng, Jiming Chen, and Taegyu Kim
(Zhejiang University, China; Pennsylvania State University, USA)
UAV planners are safety-critical software components whose behavior depends on high-dimensional, continuously tunable parameters. While existing fuzzing approaches focus on discovering failures, configurations deep inside failure regions are often diagnostically uninformative.Once the planner moves far beyond the safety boundary, parameter sensitivity diminishes, and inter-parameter dependencies become obscured.We refer to this phenomenon as diagnostic saturation and observe that configurations near the safe-unsafe transition preserve informative parameter sensitivity and interaction structure, making them more suitable for root-cause analysis.
To this end, we propose Planner-Fuzz, a boundary-oriented fuzzing framework. It steers search toward a narrow band around the safety boundary while preventing convergence to a single configuration to enable the exploration of diverse boundary behaviors. Using the collected boundary configuration, Planner-Fuzz performs three-level diagnosis through sensitivity analysis, interaction mining, and counterfactual validation to identify and validate the parameters responsible for failure behavior. We evaluate the approach on three open-source UAV planners under four anomaly types. The results show that Planner-Fuzz discovers more diverse and diagnostically useful boundary configurations than failure-driven baselines, provides stronger signals for identifying influential parameters and interactions, and yields stronger intervention-based recovery evidence. These findings suggest that fuzzing for safety-critical planners should go beyond failure discovery and explicitly support diagnosis.

Article Search Article: ase26main-p1399-p doi:10.1145/3832783.3834389

Automated Assertion Generation and Regression Testing for Machine Learning Notebooks
Yingao (Elaine) Yao, Vedant Nimje, Varun Viswanath, and Saikat Dutta
(Cornell University, USA; Veermata Jijabai Technological Institute, India; Dwarkadas J Sanghvi College of Engineering, India)
Jupyter Notebooks have become the de-facto choice for data scientists and machine learning (ML) engineers for prototyping and experimenting with ML pipelines. Notebooks provide a rich interactive interface with support for code, data, and visualization in one place. However, notebooks provide limited support for testing. As a result, during continuous development, many silent (non-crashing) regressions often go unnoticed and make notebooks unreliable and results hard to reproduce. To enable more systematic testing of ML notebooks, we introduce NBTestGen – the first automated assertion generation approach for ML notebooks. NBTestGen generates regression-based assertions that check properties related to data processing, model building, and model evaluation steps in a typical ML notebook. To support systematic integration of such assertions in notebooks, we introduce the first regression testing framework (called NBTest) that can be used as a Jupyter plugin and allows developers to write cell-scoped assertions in notebooks. Three key features of such assertions are that they are 1) cell-scoped: they are linked to specific notebook cells and execute only after those cells are executed, 2) non-intrusive: they do not block notebook execution (in a Jupyter session), so that development can continue when they fail, and 3) they integrate with pytest and CI pipelines, allowing developers to easily do regression testing of their notebooks. We evaluate NBTestGen on a corpus of 585 notebooks from the popular Kaggle platform. NBTestGen generates a total of 21542 assertions (36.82 on average per notebook). The generated assertions kill 72.21% of ML-specific mutations, while maintaining a high passrate of 100.00%. We also show that NBTestGen can detect 69.68% of historical regressions in 531 older versions of Kaggle notebooks. A popular ML library, SHAP, integrated NBTest into their CI. Further, we perform a user study with 17 ML developers that shows that such users find NBTest highly intuitive and useful.

Article Search Artifacts Available Article: ase26main-p1451-p doi:10.1145/3832783.3834390

Learning the Lexical Structure of Black-Box Systems by Parsing Systematic String Edit Mutations
Moeketsi Raselimo, Lars Grunske, and Bernd Fischer
(Humboldt University of Berlin, Germany; Stellenbosch University, South Africa)
We describe and evaluate an approach to learn the lexical structure of black-box systems from a set of positive training examples. Our core idea is to use the system as oracle for repeated membership queries which we construct by systematically mutating the training examples using different string edit operations. We then learn the elements of the lexical structure by generalizing from the oracle responses to the mutated training examples.
We have applied this approach to several systems with target languages of varying complexity, such as TinyC, JSON, AMPL, Modula-2, Oberon, Zig, or C. We show that the learned tokens are similar, and in many cases identical, to those of the oracle lexers. Their inspection can indicate where the systems under test deviate from the lexical structure of their respective target languages; the learned tokens can thus serve as precise test oracles for lexers. We show that the segmentations induced by lexing the validation sets with the learned and oracle lexers, respectively, are identical in many cases and that the non-identical parts affect on average between zero and about 15 percent of the characters. We finally demonstrate that using the learned tokens substantially improves the quality of the context-free grammar that the state-of-the-art grammar learner Kedavra learns for cJSON, increasing the average F1-score of the grammars learned in five runs from 0.93 to 1.00.

Article Search Article: ase26main-p1479-p doi:10.1145/3832783.3834391

KQFuzz: Knowledge-Guided Fuzzing for Quantum Libraries via Large Language Models
Fuyuan Xia, Qixin Zhang, Chenhao Ying, Haojin Zhu, Shuai Wang, Yuan Luo, Pingchuan Ma, and Yuxuan Du
(Shanghai Jiao Tong University, China; Nanyang Technological University, Singapore; Hong Kong University of Science and Technology, Hong Kong; Zhejiang University of Technology, China)
As quantum computing continually improves, ensuring the reliability and correctness of quantum libraries has become increasingly critical. To this end, many LLM-based fuzzing approaches towards quantum libraries have been proposed to uncover potential bugs. However, these methods still suffer from limitations such as insufficient flexibility and low efficiency, which hinder the progress of the quantum computing field. To address these challenges, we propose KQFuzz, a novel knowledge-guided fuzzer for quantum libraries. It leverages comprehensive codebase knowledge to ground LLM-based test generation, synergizing this with fitness-guided evaluation and two-level mutations to explore complex execution paths and trigger potential bugs. Firstly, KQFuzz introduces a novel prompting scheme tailored to quantum programs, which strategically incorporates knowledge of the codebase to efficiently generate high-quality quantum seed programs. Moreover, we develop evaluation and mutation strategies to handle the generated seed programs, facilitating efficient fuzzing execution while further enriching the diversity of the resulting test cases. We implement KQFuzz and conduct fuzzing on three popular quantum libraries, including Qiskit, PennyLane, and Cirq. Experimental results demonstrate that our approach significantly outperforms other state-of-the-art methods, with coverage improved by up to 18.44%. During the development of KQFuzz, we discovered 13 bugs, all of which have been confirmed and 12 have already been fixed by the developers.

Article Search Article: ase26main-p1509-p doi:10.1145/3832783.3834392

What Breaks When LLMs Code? Characterizing Operational Safety Failures of Agentic Code Assistants
Alif Al Hasan and Sumon Biswas
(Case Western Reserve University, USA)
Autonomous coding agents built on large language models (LLMs) are rapidly being integrated into development workflows, yet their operational safety properties remain poorly understood beyond evaluations of explicitly malicious inputs. In practice, high-impact failures arise during benign, goal-directed use through environment breakage, fabricated success reports, etc. that current benchmarks do not capture. What categories of operational safety failures actually occur when coding agents are used for everyday development tasks and what is their impact? We present an incident-driven empirical study grounded in two complementary evidence streams. We screen 68,816 papers from 22 premier venues, curating 185 safety-relevant studies, and mine 16,586 GitHub issues from widely deployed LLM-powered coding tools, manually confirming 547 genuine safety failures. Applying systematic open coding over both corpora, we derive a multi-dimensional safety taxonomy of 33 operational risk types organized across seven dimensions, and annotate each incident with contributing factors, task context, severity, and downstream impact. Our findings show that coding-agent failures are often severe, with 326 of 547 incidents rated high or critical. The dominant risks are constraint violations, destructive operations, authorization bypasses, and deception, and over 65% of incidents arise in bug fixing and setup or configuration, patterns largely missing from prior literature. These results have direct implications for SE tool designers and benchmark developers: guardrails must go beyond adversarial-prompt defenses to enforce environmental constraints, failure transparency, and safe-halt behaviors.

Article Search Artifacts Available Article: ase26main-p1517-p doi:10.1145/3832783.3834393

When Compression Becomes an Attack Surface: Black-Box Attacks on Prompt-Compressed LLM Agents
Zesen Liu, Zhixiang Zhang, Yuchong Xie, and Dongdong She
(Hong Kong University of Science and Technology, China; Fudan University, China)
Prompt compression is increasingly deployed in LLM agents to reduce latency and cost. It determines what the backend LLM ultimately sees, thereby unexpectedly changing the security boundary of the agent pipeline. We show that, when trusted and untrusted inputs are compressed under a shared budget, this lossy transformation exposes a new attack surface in the LLM agents. By perturbing only untrusted inputs before compression, an adversary can cause the compressor to discard task-critical evidence or safety guardrails before LLM inference. Unlike prompt injection, jailbreaks, or RAG poisoning, our attack target is the compressor rather than the backend LLM. The perturbation need not encode a meaningful instruction or even survive compression. We formalize this vulnerability as adversarial information loss (AIL), the excess downstream behavioral distortion caused by adversarially steering a lossy compressor beyond benign compression alone. To exploit AIL, we present COMA, a transfer-based black-box attack that uses attacker-side surrogate compressors and backend LLMs to optimize pre-compression perturbations. COMA first selects a misbehavior-inducing target in compressed space, then searches for a perturbation whose surrogate compression matches that target. Across three tasks (agent tool selection, question answering, and system prompt corruption) and six widely used compressors, COMA achieves an average attack success rate of 0.71, versus 0.21 for the strongest non-compression-based baseline. COMA further generalizes across compression budgets, backend LLMs, and surrogate compressors, transfers to two case studies of real-world agents. We also show that isolating trusted inputs from untrusted inputs during compression improves robustness, especially for system prompt corruption.

Article Search Artifacts Available Article: ase26main-p1531-p doi:10.1145/3832783.3834394

ExplainBench: Evaluating Code Explanations from Agents
Zhiyuan Pan, Sungmin Kang, Imam Nur Bani Yusuf, and Abhik Roychoudhury
(Zhejiang University, China; National University of Singapore, Singapore)
Large Language Model (LLM) agents have seen rapid adoption in software engineering. As agents take a greater role in the actual generation of code, they are making larger changes, spanning tens to hundreds of lines. This makes manual review of agent results increasingly infeasible, leading developers to turn to explanations to understand enacted changes. Despite this, there are no benchmarks that evaluate the trustworthiness of agent-generated explanations. To bridge this gap, we propose ExplainBench, a benchmark to automatically evaluate explanations from coding agents. ExplainBench is based on the intuition that informative explanations should enable an LLM to correctly answer questions, allowing quantitative comparison of explanation quality between agents. With this observation, we construct a suite of questions that evaluates whether explanations accurately describe (1) the intended behavior of buggy code and (2) the effect of applying the agent patch itself. Experiments first reveal that explanation quality is a distinct axis of agent evaluation: ExplainBench ranks agents differently from the widely-used SWE-bench Verified benchmark. A deeper breakdown of explanation quality in agents shows frequent problems in explanations, such that explanations often claim that a patch is correct when it is not. Based on this insight, we implement and evaluate an explanation audit agent which runs additional tests to validate and refine explanations. This agent improved the explanations of all evaluated agents, demonstrating agent explanations can be automatically made more trustworthy.

Article Search Article: ase26main-p1571-p doi:10.1145/3832783.3834395

Finding Typing-Related Bugs in JVM Implementations
Yanzhou Lu, Junjie Chen, Yingquan Zhao, Tianchang Gao, and Zan Wang
(Tianjin University, China)
As an object-oriented language, Java relies heavily on its type system. Existing testing efforts mainly target type-related implementations in the Java compiler, while the substantially different type-related implementations in the Java Virtual Machine (JVM) remain insufficiently tested. Although many techniques have been proposed for direct JVM testing, they mostly focus on specific components such as the JIT compiler or garbage collector, leaving type-related implementations underexplored. In this paper, we present Caster, the first technique for testing JVM type-related implementations by directly generating bytecode test programs guided by type specifications in the Java Language Specification (JLS). Specifically, Caster designs a set of feature primitives to encapsulate the operations needed for type-related testing, and leverages a Large Language Model (LLM) to identify type-related test scenarios from the JLS and guide primitive composition. Based on these scenarios, Caster organizes the selected primitive combinations into a multi-unit test-program generator, whose Cartesian-product expansion produces a large number of concrete test programs. To support large-scale execution, Caster dynamically loads these generated programs from memory and invokes them via reflection across different JVM implementations, collects their result lists, and compares them to identify inconsistencies and detect potential JVM bugs. Extensive experiments on four Long-Term Support OpenJDK versions of three popular JVM implementations (i.e., HotSpot, OpenJ9, and Dragonwell) demonstrate that Caster outperforms the state-of-the-art techniques in detecting more JVM inconsistencies. Moreover, Caster has uncovered a total of 21 previously unknown defects in the latest JVM releases, among which 18 have been confirmed or fixed by developers.

Article Search Article: ase26main-p1588-p doi:10.1145/3832783.3834396

Compressing Code Context for LLM-Based Issue Resolution
Haoxiang Jia, Earl T. Barr, and Sergey Mechtaev
(Peking University, China; University College London, UK)
Large Language Models (LLMs) are now capable of resolving real-world GitHub issues. However, current approaches overapproximate the code context and suffer from two compounding problems: the prohibitive cost of processing massive inputs, and low effectiveness as noise floods the context window and distracts the model from the bug-fixing signal. Existing compression techniques fail to resolve this tension: generic compressors compromise the semantic integrity of code, while code-specific tools lack awareness of code structure and task context to preserve essential patch ingredients. To address this, we propose a novel framework consisting of two components. First, Oracle-guided Context Distillation (OCD), a context distillation algorithm that combines genetic search and delta debugging to systematically reduce code contexts to their minimal sufficient subsequence — retaining only the ingredients required for a successful fix. We use this distilled data to fine-tune , a lightweight model that learns to compress code context at inference time, filtering noise and combating distraction while preserving fix ingredients. Evaluated on SWE-bench Verified across three frontier LLMs, maintains a stable compression rate of about 6× across models, reduces the total token budget by 51.8%–71.3% relative to the uncompressed setting, improves issue resolution rates by 5.0%–9.2%, and delivers the best overall balance among effectiveness, compression ratio, and latency compared with state-of-the-art context compression baselines.

Article Search Article: ase26main-p1597-p doi:10.1145/3832783.3834397

When Dependencies Depend: A Large-Scale Study on Conditional Dependencies and Their Security Implications in C/C++
Yifeng Di, Juewei Zhang, and Tianyi Zhang
(Purdue University, USA; Carnegie Mellon University, USA)
Third-party libraries (TPLs) are a major source of software supply chain risk. Yet accurately managing them remains difficult in C/C++, where dependency inclusion is often encoded directly in build scripts rather than using standard package managers. A particularly overlooked challenge is conditional dependencies, i.e., TPLs included only under specific build configurations. C/C++ projects commonly express such conditions through fine-grained CMake logic involving platform checks and user-defined configuration variables, making them difficult for existing Software Composition Analysis (SCA) tools to reason about. To investigate this problem, we conduct a large-scale empirical study of 2K real-world C/C++ projects. We develop Condor, a static analysis approach that parses CMake build files to extract conditional dependencies together with their inclusion logic. Our results show that conditional dependencies are pervasive: among projects with at least one detected TPL, 84.33% contain at least one conditional dependency, and on average 53.77% of declared TPLs are conditionally included. We further find that 87.86% of conditional dependencies are governed by user-defined variables, revealing a C/C++-specific landscape of dependency variability that is difficult for existing SCA tools to capture. Finally, our evaluation of four popular SCA tools shows that up to 48.33% of reported vulnerable dependencies are conditionally included and may become false alarms if not included in a given build configuration. These findings highlight a key limitation of current SCA practice and motivate configuration-aware vulnerability analysis for C/C++ projects.

Article Search Article: ase26main-p1654-p doi:10.1145/3832783.3834398

Towards Fully Automated Medical Imaging Code Generation via Validation-Based Context Engineering
Zixiao Zhao, Jing Sun, Zhe Hou, Cheng-Hao Cai, Qian Liu, Mengze Li, Zijian Zhang, and Jin Song Dong
(University of Auckland, New Zealand; Griffith University, Australia; Monash University, Suzhou, China; Beijing Institute of Technology, China; National University of Singapore, Singapore)
Large language models (LLMs) have demonstrated considerable promise in program generation for small-scale and conventional application development; however, they remain limited when applied to complex, domain-specific tasks such as medical image processing. General-purpose models lack explicit domain knowledge and robust validation mechanisms to ensure correctness, often requiring substantial human intervention to produce reliable processing pipelines. To address these limitations, we propose AutoMedImg, a multi-agent framework for fully automated medical image processing code generation. AutoMedImg orchestrates specialised agents across two phases: a Planning Phase that performs dataset analysis and architecture design with semantic and formal verification, and a Coding Phase that generates modules in parallel with static checking, execution testing, and assembly validation. This multi-stage validation mitigates error propagation throughout generation, while comprehensive auto-context engineering combining domain-specific knowledge bases, shared memory, and validation feedback automates context construction without manual prompting. A cross-project adaptive pipeline synthesis mechanism further accumulates validated pipelines and retrieves proven components for new tasks based on project similarity, enhancing generation efficiency through cross-project learning. Extensive evaluation across six diverse and well-established medical imaging datasets with five backbone LLMs demonstrates that AutoMedImg achieves zero human intervention, with Dice scores of up to 0.90 for segmentation tasks and 99% accuracy for classification.

Article Search Article: ase26main-p1668-p doi:10.1145/3832783.3834399

From Plan to Action: How Well Do Agents Follow the Plan?
Shuyang Liu, Saman Dehghan, Jatin Ganhotra, Martin Hirzel, and Reyhaneh Jabbarvand
(University of Illinois at Urbana-Champaign, USA; IBM, USA)
Agents are commonly instructed to follow a task-specific plan for guidance.However, it is unknown to what extent agents actually follow instructed plans. Without such an analysis---determining the extent agents comply with a given plan---it is impossible to assess whether a solution was reached through correct strategic reasoning or through other means, e.g., data contamination or overfitting to a benchmark. This paper presents the first extensive, systematic analysis of plan compliance in programming agents, examining 21,120 trajectories from SWE-agent across four LLMs on SWE-bench Verified and SWE-bench Pro under eight plan variations. Without an explicit plan, agents fall back on internalized workflows during training, which are often incomplete, overfit, or inconsistently applied. Providing the standard plan improves issue resolution, and we observe that periodic plan reminders can mitigate plan violations and improve task success. A subpar plan hurts performance even more than no plan at all. Surprisingly, inserting additional task-relevant phases in the early stage can degrade performance, particularly when these phases do not align with the model’s internal problem-solving strategy. These findings call for fine-tuning paradigms that teach models to follow instructed plans, rather than encoding task-specific plans in them, so that they reason and act adaptively, rather than memorizing workflows.

Article Search Article: ase26main-p1675-p doi:10.1145/3832783.3834400

Needle in the Repo: Diagnosing Maintainability Failures in AI-Generated Repository Edits
Haichao Zhu, Qian Zhang, Jiyuan Wang, Zhaorui Yang, and Yuxin Qiu
(Reality Vison, USA; University of California at Riverside, USA; Tulane University, USA)
AI coding agents can now complete complex programming tasks, but existing evaluations largely emphasize behavioral correctness and often overlook maintainability risks such as weak modularity or testability. Automation without accountability shifts invisible cost downstream because hidden maintainability failures incurred today are deferred to the developers who must extend, debug, and sustain these systems tomorrow, raising significant technical and organizational concerns for software engineering.
We present Needle in the Repo (NITR), a diagnostic, probe-and-oracle framework for evaluating whether behaviorally correct repository edits preserve maintainable structure. The key idea is to distill recurring software engineering wisdom into controlled probes embedded in small, realistic multi-file codebases, with each probe designed so that success depends primarily on one targeted maintainability dimension. Each probe is paired with a hidden evaluation harness that combines (i) functional tests for required behavior, and (ii) structural oracles that encode the targeted maintainability constraint and return interpretable diagnoses.
Using NITR, we extensively evaluate 23 coding configurations across GPT, Claude, Gemini, and Qwen families in both direct-inference and agent-based settings. We find that current AI coding systems remain far from robust. On average, all AI coding configurations solve only 36.2% of cases, the best reaches 57.1%, and performance drops from 53.5% on micro cases to 20.6% on multi-step cases. The hardest pressures are architectural rather than local edits, especially dependency control (4.3%) and responsibility decomposition (15.2%). Moreover, 64/483 outcomes (13.3%) pass all functional tests yet fail the structural oracle. Under our harness, agent-mode configurations improve average performance from 28.2% to 45.0%, but do not eliminate the architectural failures.
These results show that progress in code generation is not yet progress in maintainable code evolution, and that NITR exposes a critical failure surface largely missed by conventional evaluation.

Article Search Article: ase26main-p1701-p doi:10.1145/3832783.3834401

An Empirical Study of Data Access Practice in Android AR Apps to Understand User Privacy Risks
Sabbir Hussain Meraj, Long Trac, Xiaoyin Wang, Xusheng Xiao, and Wei Wang
(University of Texas at San Antonio, USA; Arizona State University, USA)
Mobile Augmented Reality (AR) applications have rapidly expanded across domains such as gaming, education, and healthcare. However, AR's reliance on continuous camera input and access to sensitive data, such as faces and the surrounding environment, raises major privacy concerns.
Adhering to best privacy practices requires applications to clearly disclose their use of sensitive data and follow the Principle of Least Privilege (PoLP). This paper presents an empirical study of 179 Android AR applications to assess their data access behaviors and evaluate whether these practices are upheld in the real world.
To this end, we developed a novel static analysis technique to examine data accesses of these apps and identify potential privacy risks. Our analysis centers on core AR components, namely anchors and trackables, which link virtual content to real-world entities. Our findings revealed that AR apps often access sensitive data without adequate disclosure and frequently access more data than their functionality requires through unrestricted SDK APIs.
Notably, our analysis identified 58 instances of undisclosed access to facial data, 2D images, and skeletal information, and 82 instances of PoLP violations through overly broad data access. A subset of these findings was further confirmed through dynamic instrumentation.
Furthermore, our analysis suggests that protecting user privacy in AR apps can benefit from fine-grained access control, AR SDK API redesign, automated disclosure verification, and control/data-flow analysis. Particularly, there are 317 common call paths among the 754 AR-related paths we analyzed, suggesting that many AR apps share similar data usage patterns that can be leveraged for efficient privacy auditing and anomaly detection with manageable overhead.

Article Search Artifacts Available Article: ase26main-p1726-p doi:10.1145/3832783.3834402

Clarity Is Not Assumed: Understanding LLM-Based Code Generation under Ambiguous Requirements
Di Yang, Xinou Xie, Xiuwen Yang, Ming Hu, Yihao Huang, Yueling Zhang, Weikai Miao, Ting Su, Chengcheng Wan, and Geguang Pu
(East China Normal University, China; Shanghai Innovation Institute, China)
Software requirement ambiguity is ubiquitous in real-world development, stemming from the inherent imprecision of natural language and the varying interpretations of stakeholders. While Large Language Models (LLMs) have demonstrated impressive capabilities in generating code from precise specifications, such ambiguity poses a significant obstacle to reliable automated code generation. Existing benchmarks typically assume clear and unambiguous requirements, leaving an empirical gap in understanding how LLMs behave when faced with the inherent uncertainty of real-world software requirements.
In this paper, we introduce Orchid, a code generation benchmark specifically designed with ambiguous requirements. It comprises 1,304 function-level tasks covering four distinct types of ambiguity: lexical, syntactic, semantic, and vagueness. Leveraging this dataset, we conduct the first systematic empirical study to evaluate the impact of requirement ambiguity on LLM-based code generation. Our results demonstrate that ambiguity consistently degrades the performance of all evaluated LLMs, with the most pronounced negative effects observed in highly advanced models. Furthermore, we observe that LLMs frequently produce functionally divergent implementations for the same ambiguous requirement and lack the capability to identify or resolve such ambiguity autonomously. These findings reveal a significant performance gap between clear and ambiguous requirements, underscoring the urgent need for ambiguity-aware techniques in the next generation of automated software engineering tools. The Orchid benchmark is publicly available at https://huggingface.co/datasets/SII-YDD/Orchid.

Article Search Article: ase26main-p1878-p doi:10.1145/3832783.3834403

Conflict Extraction in Probabilistic Datalog Analyses
Siyu Chen, Chungha Sung, Xuyang Li, and Jingbo Wang
(Purdue University, USA; Amazon Web Services, USA)
Probabilistic extensions of Datalog enable static analyses such as pointer analysis, data race detection, and side-channel analysis to rank alarms by likelihood, but this added expressiveness also introduces a new challenge absent from deterministic analyses: the final output may contain alarms that are individually plausible yet mutually inconsistent, because marginal probabilities do not guarantee joint satisfiability. As a result, developers may spend effort investigating combinations of alarms that can never co-occur in any possible world. We address this problem by formalizing such inconsistencies as minimal unsatisfiable subsets (MUSes) and introducing PPProbe, a conflict extractor specialized for probabilistic Datalog analyses. Rather than improving MUS enumeration in general, PPProbe exploits the structure of Datalog derivation graphs to guide the search toward likely conflicts and prune the search space through bottom-up UNSAT inference. We evaluate PPProbe on 70 benchmarks from power side-channel analysis, data race detection, semantic diffing, and Bayesian-network inference. The results show that PPProbe achieves 2.5–24× higher throughput than state-of-the-art MUS enumerators, and that the conflicts it identifies yield a conservative estimate of false-positive reduction, filtering out an average of 47.7% of mutually inconsistent alarms.

Article Search Article: ase26main-p1888-p doi:10.1145/3832783.3834404

EMERGE: Checking Implementation Equivalence for LLMs via Rule Synthesis
Qi Zhan, Xing Hu, Xin Xia, and Shanping Li
(Zhejiang University, China)
Verifying whether two implementations of the same large language model are equivalent across frameworks is difficult in practice. Even when they implement the same computation, the graphs may differ in operator decomposition, tensor layout, and the use of fused or opaque kernels, making manual rewrite rules hard to build and maintain. We present Emerge, a framework for checking Implementation Equivalence over computation graphs extracted from model implementations. Instead of writing rules manually, Emerge infers candidate relations from execution values, and synthesizes rewrite rules on demand when existing rules are insufficient. Each synthesized rule is then validated using SMT solving when possible and constraint-aware randomized testing for opaque kernels. As these local equivalences accumulate, Emerge eventually establishes whether the two implementations are equivalent. Our implementation targets inference computation graphs captured from HuggingFace Transformers and vLLM. Our evaluation shows that Emerge establishes equivalence for correct implementation pairs at practical cost. It also detects 10 of 13 known implementation bugs, uncovers 8 previously unknown implementation issues that were later confirmed by developers, and localizes the first mismatch when equivalence cannot be established.

Article Search Article: ase26main-p1924-p doi:10.1145/3832783.3834405

SpecTrum: Specification-Guided Differential Fuzzing for Ethereum Consensus Clients
Seokhun Jeong, Gyeongmin Dan, Sukyoung Ryu, and Sungjae Hwang
(KAIST, Republic of Korea; Sungkyunkwan University, Republic of Korea)
Ethereum's consensus safety relies on independent consensus client implementations agreeing on every state transition. When they diverge due to implementation errors, the network can fork, finality can stall, and severe attacks are possible. To prevent such consensus divergences, Ethereum provides a Python reference implementation (consensus-spec), which acts as a specification, and a hand-crafted official test suite (spectests). However, as an executable implementation, Ethereum's specification defines validity implicitly through runtime behavior. As a result, it lacks a systematic way to ensure that all validity conditions are thoroughly evaluated.
We present SpecTrum, a framework that addresses this problem in three stages. First, we introduce Consensus-SpecTec, a mechanized specification of the Ethereum consensus algorithm, which makes validity conditions explicit as if-premises. Second, we define premise coverage, a metric that measures which if-premises are evaluated to true and false across spectests. Third, we develop a specification-based test generator that extracts constraints on premises not evaluated to false by spectests and generates inputs to evaluate them. Applying SpecTrum to five major Ethereum consensus clients, we identify 27 cross-client divergence cases, 22 of which cannot be found without the premises inserted in our mechanization. All 27 cases reproduce across fork versions, and extending the mechanized specification to a new fork takes modest effort proportional to the specification difference.

Article Search Artifacts Available Article: ase26main-p1970-p doi:10.1145/3832783.3834406

Where Does Balance Break? Boundary Discovery for Game Balance Testing under a Finite Simulation Budget
Hiroki Mukai, Yusaku Kato, Norihiro Yoshida, Erina Makihara, and Katsuro Inoue
(Ritsumeikan University, Japan)
Software testing often relies on assumptions such as reproducible executions and stable correctness criteria. However, many modern software systems exhibit non-deterministic executions and large behavior spaces, making exhaustive exploration impractical and single-run judgments unreliable. These characteristics make it difficult to identify where acceptable behavior ends and problematic behavior begins. Competitive multiplayer games represent a challenging instance of such systems, where balance must be maintained so that no single strategy dominates. Even small parameter changes can trigger abrupt balance disruption, yet detecting such failures requires repeated simulations under non-deterministic outcomes and high-dimensional parameter spaces. In this paper, we formulate game balance regression testing as a boundary-discovery problem under a finite simulation budget. The objective is to efficiently identify inputs near the boundary that separates balanced and unbalanced regions. To address this problem, we propose BBExplorer, which combines multi-directional candidate generation, budget-aware two-stage screening, and adaptive step-size shrinkage for boundary refinement. Experimental results on two games with different levels of complexity show that the approach is strong in low-dimensional settings and remains effective in higher-dimensional ones. It also exhibits stable boundary behavior across unseen random seeds and threshold settings. These results indicate that BBExplorer is effective for practical balance regression testing and, more broadly, for boundary-oriented testing in non-deterministic, budget-constrained systems.

Article Search Article: ase26main-p2013-p doi:10.1145/3832783.3834407

On the Effects of Customized Configurations of Static Code Analysis Tools: A Prospective Cohort Study of SonarQube Cloud
Sabato Nocera, Sira Vegas, and Giuseppe Scanniello
(University of Salerno, Italy; Polytechnic University of Madrid, Spain)
Static Code Analysis (SCA) tools are used to detect issues and enforce code quality standards in software projects. SCA tools like SonarQube Cloud provide predefined configurations that project owners can customize depending on the goal of the project at hand. One such configuration is the quality gate, which defines the set of conditions that source code must satisfy to pass automated quality checks during software development. This paper investigates the impact of customizing SonarQube Cloud quality gates on security, reliability, and maintainability. We employ a prospective cohort study design to explore causal relationships using observational data. We track 230 open-source GitHub projects forward in time, monitoring transitions in configurations to ensure consistent adherence to either customized or predefined ones. A significant effect was observed only for security under less strict configurations, suggesting that developers could be more motivated to fix security issues when the requirements are perceived as attainable. For reliability and maintainability customizations, we speculate that the lack of impact may stem from these quality attributes being perceived as less essential or pressing compared to security.

Article Search Article: ase26main-p2032-p doi:10.1145/3832783.3834408

RefineAct: Automatic Runtime Verification of LLM Agent Actions
Fraol Batole, Foutse Khomh, and Hridesh Rajan
(Tulane University, USA; Polytechnique Montréal, Canada)
LLM-based agents that invoke external tools can cause irreversible harm when their actions diverge from user intent, from deleting critical files to exposing private data or executing untrusted code. Unlike traditional software, where test suites and contracts define expected behavior, LLM agents operate without a specification: there is no artifact against which to check their actions before execution. Existing safeguards rely on prompt engineering or post-hoc evaluation, neither of which closes this gap. We present RefineAct, a runtime verification framework that automatically derives a task-specific specification from the user’s natural-language instruction and enforces it during agent execution. RefineAct operates in three stages: (1) intent formalization translates instructions into first-order Prolog predicates capturing the user’s goal and safety requirements; (2) refinement-based planning decomposes the specification into a concrete action plan with pre-/postcondition chains and risky-action patterns; and (3) runtime verification intercepts each agent-proposed action, queries the Prolog knowledge base to verify precondition satisfaction and constraint compliance, and returns a disposition state that determines whether the action proceeds, requires user confirmation, or is blocked with corrective feedback. We evaluate RefineAct on 144 agent tasks across five domains from the ToolEmu benchmark. RefineAct reduces failure incidence from 77% to 39% while improving task completion quality from 1.0 to 1.9 on a 0–3 scale, demonstrating that deriving and enforcing specifications at runtime can improve both safety and helpfulness of LLM agent behavior.

Article Search Article: ase26main-p2301-p doi:10.1145/3832783.3834409

Verifier-in-the-Loop LLM Solving of Heap Constraints for Concolic Execution
Shaoran Xia, Leyi Cheng, Caihua Dong, Dongdong She, Bo Wang, Xin Peng, and Zhen Dong
(Fudan University, China; Soochow University, China; Hong Kong University of Science and Technology, China; Beijing Jiaotong University, China)
Concolic execution systematically explores program paths by solving symbolic constraints, but in object-oriented software its practical effectiveness is often limited by heap constraints over objects and heap structure, such as nullness, aliasing, runtime types, and field reachability. These constraints form a systematic bottleneck in real-world software because satisfying them requires constructing a concrete heap model rather than merely computing primitive valuations, which existing SMT-based concolic engines and symbolic heap techniques do not handle effectively. We present COHEC, a hybrid concolic architecture that decomposes each target path condition into a primitive component solved by an SMT solver and a heap component handled by a Heap Constraint Solver (HCS). Given extracted heap constraints, HCS uses an LLM to propose candidate heap models and initialization code, while a trusted verifier executes and verifies the generated code, admitting only models whose initialization code constructs states that satisfy all constraints and keeping the LLM outside the trusted computing base. We implement COHEC on top of JDart/JPF and evaluate it on 6,750 APIs from ten Java libraries, together with a stratified sample from commons-lang3 against the LLM-assisted baseline CONCOLLMIC. The results show that COHEC substantially improves path exploration and concrete valuation generation over native JDart, while achieving higher coverage than CONCOLLMIC at lower time and token cost. Overall, the findings indicate that treating heap constraint solving as verified heap model construction is a practical and effective way to extend concolic execution to the object-sensitive behaviors pervasive in modern software.

Article Search Artifacts Available Article: ase26main-p2302-p doi:10.1145/3832783.3834410

RealAIGC: Towards More Realistic Evaluation of AI-Generated Code Detection in Real-World Code Repository
Wentao Chen, Yufei Chen, Huiqun Yu, Guisheng Fan, Kaiwen Zhi, Peite Guo, and Jianyuan Yu
(East China University of Science and Technology, China; Shanghai Engineering Research Center of Smart Energy, China)
AI-generated code (AIGC) from autonomous coding agents are widely adopted across the software development lifecycle. However, AIGC also introduces critical challenges to open-source community governance, making AIGC detection important. Existing AIGC detection benchmarks are fundamentally misaligned with real-world development processes. To address this gap, we construct RealAIGC, a benchmark for AIGC detection grounded in real-world code repositories. RealAIGC is built from the development histories of GitHub and includes pull request (PR) generated by five coding agents alongside human-written code, covering eight major programming languages. It supports multi-granularity detection at the repository, PR, commit, file, and function levels, with more than four million functions. Based on RealAIGC, we conduct a realistic empirical evaluation of 28 AIGC detection methods across three categories. The results show that supervised approaches perform well only in small, homogeneous, single-language repositories, but degrade significantly in mainstream dynamic languages and in large-scale, multi-developer environments. Perplexity based methods are nearly ineffective in realistic settings. No existing method demonstrates general applicability, and all approaches perform unsatisfactorily at the commit and PR levels. This work provides empirical guidance for advancing AIGC detection techniques.

Article Search Article: ase26main-p2317-p doi:10.1145/3832783.3834411

Hybrid Fuzzing in Input Generation Logic
Siwei Wei, Shihao Zhu, Ruijie Meng, and Yan Cai
(Institute of Software at Chinese Academy of Sciences, China; University of Chinese Academy of Sciences, China; National University of Singapore, Singapore)
The core challenge of fuzzing is to consistently generate test inputs that are both syntactically valid and semantically diverse. However, mutation-based coverage-guided fuzzing often struggles to achieve this, especially for highly structured formats. Two lines of work have emerged to address this challenge: (1) hybrid fuzzing uses constraint solving to drive execution toward hard-to-cover program states to increase semantic diversity, and (2) generator-based fuzzing relies on predefined or synthesized input-generation programs to ensure syntactic validity. Both approaches, however, have significant limitations. (1) In hybrid fuzzing, solved inputs are fed back into the fuzzing queue for byte-level mutation, which often breaks syntactic structure (e.g., checksums and cross-field dependencies) and causes early rejection. (2) In generator-based fuzzing, semantic diversity is constrained by static generators based on domain knowledge; although recent work introduces generator mutations, they remain blind and therefore have limited adaptability to the target program’s logic.
In this paper, we present hybrid generator space fuzzing, a method that combines the strengths of hybrid fuzzing and generator-based fuzzing to improve the syntactic validity and semantic diversity of test inputs. The key idea is to treat input generators, rather than test inputs, as the unit of targeted solving, and to use the LLM as constraint solver to derive new generators. This design offers two benefits. First, compared with hybrid fuzzing, it preserves syntactic validity because inputs are generated directly from evolving generators rather than through byte-level mutation of solved seeds. Second, compared with generator-based fuzzing, it overcomes the limitations of static generators and blind generator mutations, enabling the fuzzer to adapt to target program logic and explore a broader semantic space. Building on this concept, we implement GenEvo, an LLM-driven hybrid generator space fuzzer. Our evaluation on a large set of real-world programs shows that GenEvo, using a locally deployed medium-sized LLM, significantly outperforms competitive baselines (including general-purpose, hybrid, structure-aware, and LLM-assisted fuzzers) in both branch coverage (with an average improvement of 98.3%) and vulnerability discovery. Furthermore, GenEvo has discovered 25 previously unknown vulnerabilities in these extensively-tested programs.

Article Search Article: ase26main-p2343-p doi:10.1145/3832783.3834412

Understanding the Code and Architectural Smells Generated by LLMs and Autonomous Agents
Yuecai Zhu, Nikolaos Tsantalis, and Peter C. Rigby
(Concordia University, Canada)
The promise of Large Language Models (LLMs) in automated software engineering is often measured by functional correctness, overlooking the critical issue of long-term maintainability. This paper presents a systematic audit of technical debt in AI-generated software, revealing that AI does not eliminate flaws but rather introduces a distinct "machine signature" of defects. Our multi-scale analysis—spanning single-file algorithmic tasks and complex, agent-generated systems—identifies a fundamental Reasoning-Complexity Trade-off: as models become more capable, they generate increasingly bloated and coupled code. This architectural decay is so pronounced that we establish a Volume-Quality Inverse Law, where code volume is a near-perfect predictor of structural degradation. Crucially, we demonstrate that neither functional correctness nor detailed prompting mitigates this decay. These findings challenge the current paradigm of prompt-driven generation, reframing the central problem of AI-based software engineering from one of code generation to one of architectural complexity management. We conclude that future progress depends on equipping agents with explicit architectural foresight to ensure the software they build is not just functional, but also maintainable.

Article Search Article: ase26main-p2377-p doi:10.1145/3832783.3834413

Toward Inclusive Programming Support for Novice Programmers with Dyslexia: Insights from a Comparative Study
Zihan Fang, Janice Chung, Ruijia Chen, Marcia Barnes, Yuhang Zhao, and Yu Huang
(Vanderbilt University, USA; University of Wisconsin-Madison, USA)
Programming attracts a diverse range of learners, including individuals with dyslexia, a learning difficulty that affects reading, writing, and spelling despite average intelligence and educational opportunities. Although prior work has examined dyslexia in natural-language and code comprehension, we still know little about how dyslexic learners engage with code relative to natural language, and what this implies for inclusive programming environments. This gap is especially important in the AI era, as novice programmers increasingly rely on AI-powered coding and learning tools that may not address the distinct comprehension and cognitive support needs of learners with dyslexia. To address this gap, we conducted an eye-tracking study with a mixed-methods design involving 24 novice programmers (12 with dyslexia, 12 without) who completed paired code-comprehension and English-comprehension tasks. We found that although the two groups showed similar overall visual scanning patterns, eye-gaze features suggest higher cognitive effort among dyslexic novices during code comprehension. We also found that the performance gap between dyslexic and neurotypical learners was smaller in code comprehension than in English comprehension, suggesting that programming-specific representations may reshape, rather than eliminate, dyslexia-related challenges. In addition, dyslexic novices reported lower confidence, viewed general training methods as less effective, and described difficulties with symbolic elements and structurally complex code. These findings provide an empirical basis for designing AI-assisted programming tools that can detect cognitive overload and adapt explanations, feedback, and task support to better support dyslexic novices.

Article Search Article: ase26main-p2412-p doi:10.1145/3832783.3834414

AEG-Edit: Bridging API Evolution and Code Semantics for Accurate API Usage in Code Generation
Yitong Lin, Haoye Tian, Dehai Zhao, Chao Ni, Xiaohu Yang, and Xiaoxue Ren
(Zhejiang University, China; Aalto University, Finland)
As APIs across standard libraries and software ecosystems evolve, the API knowledge frozen in pre-trained LLMs becomes outdated. Prompt-based methods supply updated documentation at inference time yet suffer from context neglect, often reverting to memorized deprecated usages. Existing parametric approaches, whether fine-tuning or model editing, face two common challenges: they treat API changes as textual fact updates, overlooking the changes in code semantics needed for executable correctness (C1), and lack a mechanism to concentrate updates on positions involved in the API evolution, causing parameter interference beyond the evolution boundary (C2). We propose AEG-Edit, which constructs a heterogeneous API evolution graph, encodes and aligns the graph signal with the LLM hidden space, and concentrates gradient effort on positions involved in the API evolution through API-weighted windowing and optimization. On RustEvo²+ (708 Rust tasks) and PyEvo (622 Python tasks) across two representative open-source LLMs at the 7–8B parameter scale, AEG-Edit achieves statistically significant Pass@1 gains on RustEvo²+, improving over the strongest baseline by up to 6.49% on Qwen2.5-7B-Instruct. Ablation and transferability analyses confirm that the heterogeneous API evolution graph drives the gains and transfers to alternative editing backbones.

Article Search Artifacts Available Article: ase26main-p2449-p doi:10.1145/3832783.3834415

ReqEvolve: User-Oriented Software Self-Evolution through Automatic Requirement Interpretation
Md Asif Iqbal Fahim and Alessio Ferrari
(University College Dublin, Ireland; National Research Council, Italy)
The paradigm of software self-evolution enables systems to autonomously extend and reconfigure their own capabilities during execution in response to technical specifications. Yet requests for new functionality often originate from end users and are rarely expressed in technical terms. As a result, developers must translate user needs into technical specifications before the system can evolve, delaying early validation of the requested functionality by preventing users from immediately observing the resulting behaviour. To address this gap, we present ReqEvolve, a runtime code generation system that enables user-driven self-evolution by accepting high-level user requests. The system integrates automatic requirements engineering (RE) and test-driven development (TDD) to transform these requests into executable functionality through clarification, specification decomposition, test generation, and runtime integration. We evaluate ReqEvolve on 72 software evolution cases across 18 projects against two baselines: SpecFix, an RE-focused code generation approach, and an ablation variant of our system. ReqEvolve achieves 89.2% Pass@1, outperforming SpecFix by 18.8% (p < 0.01, r = 0.79, large effect) and the ablation baseline by 32.6% (p < 0.001, r = 0.88, large effect). These results provide initial evidence that user-driven self-evolution is a viable paradigm for autonomously extending software capabilities from user requests, thereby accelerating requirements validation prior to developer verification.

Article Search Article: ase26main-p2528-p doi:10.1145/3832783.3834416

Testing IFrame Rendering in Web Browsers
Dahyeon Park, Yuseok Jeon, Seongil Wi, and Mijung Kim
(UNIST, Republic of Korea; Korea University, Republic of Korea)
Iframes are widely used in modern web development, but their separation from the parent page can lead to iframe-specific rendering bugs, which occur only when content is embedded within an iframe. These bugs may degrade user experience and, in some cases, lead to serious security vulnerabilities. However, existing rendering bug detection techniques are largely ineffective since their oracles compare rendering outputs only within the same context and fail to capture discrepancies across different rendering contexts.
This paper presents ICheck, an automated testing technique for iframe rendering in web browsers. Specifically, given an HTML document, ICheck introduces a cross-context oracle that detects inconsistencies by comparing the same HTML content in both top-level and iframe contexts. To further assist debugging, ICheck performs layout-aware input reduction by redefining the effective search space of bug-inducing elements. As a result, ICheck helps developers more effectively and efficiently identify and diagnose iframe-specific rendering bugs. Evaluated on the most popular browsers, Chrome and Firefox, ICheck detected 21 previously unknown bugs, all of which were confirmed. On average, ICheck uncovered 3.8× more bugs than state-of-the-art rendering bug detection work. In addition, ICheck effectively reduced HTML inputs by 93% in DOM nodes while achieving up to 10.1× faster reduction time compared with state-of-the-art delta debugging approaches.

Article Search Artifacts Available Article: ase26main-p2611-p doi:10.1145/3832783.3834417

Real-World Perturbation Testing of Autonomous Driving Systems
Stefano Carlo Lambertenghi, Matthias Weil, and Andrea Stocco
(TU Munich, Germany; fortiss, Germany)
Autonomous Driving Systems (ADS) must operate reliably under diverse conditions, yet representative data for rare or adverse scenarios is difficult to obtain. Perturbation-based testing is widely used to assess robustness, but most studies focus on offline datasets or simulation, leaving open questions about how such results translate to real-world driving. We present a large-scale study of 72 camera and LiDAR perturbations, evaluated across three testing modalities: offline model-level analysis, hardware-in-the-loop execution, and closed-loop system-level testing on a full-scale autonomous vehicle. The study covers both an end-to-end vision-based driving model and a modular LiDAR-based perception and planning stack. Our results reveal a clear gap between testing levels. For camera-based systems, perturbations with limited offline impact can still induce unstable control and failures in real-world driving. For LiDAR-based systems, degradation is more consistent at the perception level but weakly predictive of system-level failures. Across both modalities, model-level metrics alone are insufficient to identify the most harmful perturbations.We further show that real-time feasibility is a key constraint in real-world testing, and that robustness observations obtained from recorded data do not consistently transfer to closed-loop behavior on a physical vehicle, highlighting the importance of complementary real-world, system-level evaluation.

Article Search Article: ase26main-p2645-p doi:10.1145/3832783.3834418

SwiftGram: Dynamic Lexical Forking for Precise and Efficient Grammar-Constrained Decoding
Shuyi Ling, Liangyi Kang, Jie Liu, Shuai Wang, Zhirou Ma, and Dan Ye
(Institute of Software at Chinese Academy of Sciences, China)
Large Language Models (LLMs) must strictly adhere to grammatical constraints for reliable structured generation in agentic workflows, code synthesis, and domain-specific language (DSL) generation. Grammar-Constrained Decoding (GCD) enforces this by masking out tokens incompatible with the target grammar at each decoding step, and has been adopted as a core module in mainstream inference engines such as vLLM. However, existing GCD methods exhibit precision defects under grammars with overlapping terminal definitions. The misalignment between LLM tokens and grammar terminals induces terminal ambiguity, where a token admits multiple plausible terminal interpretations. Existing methods handle this imprecisely, causing both false acceptance and false rejection, while incurring high computational overhead. We propose SwiftGram, a GCD framework that incorporates dynamic lexical forking for precise token masking. By forking only at detected ambiguity boundaries, SwiftGram preserves all admissible terminal interpretations while suppressing spurious ones. It builds on Generalized LR (GLR) parsing with the Graph-Structured Stack (GSS) for compact multi-branch parsing, together with adjacency pruning and terminal tries to reduce online overhead. Experiments show that SwiftGram achieves near-zero grammar violations and no observed false rejection, whereas strong baselines remain prone to masking errors. SwiftGram improves effective throughput by up to 14.7% over the strongest baseline under rejection sampling, and achieves a 5.9× average speedup in per-step mask construction under mask-before-selection.

Article Search Article: ase26main-p2654-p doi:10.1145/3832783.3834419

GraphLedger: Repository-Level Vulnerability Detection via Graph-Based Compression and Assumption Validation
Thi-Hong-Cuc Le, Hoang-Quoc-Bao Hua, and Xuan-Bach Le
(Ho Chi Minh City University of Technology, Vietnam)
Repository-level vulnerability detection requires reasoning across files, procedures, and long-range dependency chains. This is difficult for both dominant families of methods: static analysis often over-approximates and produces many false positives, while LLM-based approaches struggle with large repository context and may generate plausible but unsupported claims.
We present GraphLedger, a framework that combines Adaptive Assumption-Corroborating Compression (AACC) with an Assumption Ledger for structured hypothesis verification. AACC reduces repository context while preserving vulnerability-relevant dependency structure. The LLM then proposes explicit vulnerability hypotheses over the compressed context, and the ledger validates their assumptions against repository evidence, producing Verified, Rejected, or Inconclusive outcomes. This makes repository-scale reasoning more auditable and less prone to unsupported conclusions.
GraphLedger targets taint-style, source–sink vulnerability hypotheses with bounded inter-procedural evidence, rather than arbitrary vulnerability classes. We conduct a controlled cross-method evaluation on CWE-Bench-Java and a complementary within-system component analysis on CVEfixes. On CWE-Bench-Java, AACC achieves 0.71 token reduction while retaining 0.90 of dependency edges and 0.88 of taint paths. On a documented, repository-disjoint 100-repository CWE-Bench-Java evaluation partition, GraphLedger achieves 0.77 precision, 0.76 recall, and 0.76 F1, and reduces false discovery rate from 0.41 to 0.23 relative to VulAgent under a controlled shared-backend evaluation. These results show that combining structure-aware compression with assumption validation improves the reliability and transparency of repository-level vulnerability detection.

Article Search Artifacts Available Article: ase26main-p2655-p doi:10.1145/3832783.3834420

Escaping the Self-Repair Trap: Improving Test Oracle Generation via Dual-Context Awareness
Kefan Li, Hongyue Yu, and Yuan Yuan
(Beihang University, China)
Large Language Models (LLMs) have shown strong potential for regression-oracle completion, where a test prefix is given and the current program version is treated as expected behavior. Recent approaches increasingly rely on iterative self-repair and execution feedback, but optimizing execution success does not necessarily yield strong fault-revealing oracles. This objective, widely adopted in repair-based methods, serves only as a proxy and may be misaligned with the true goal of oracle generation. Such misalignment biases the repair process, giving rise to a feedback-driven degeneration that we term the Self-Repair Trap, where iterative repair progressively drives models toward assertions that are easier to satisfy but less effective at detecting faults. To address this issue, we propose DCAware, a computationally efficient, non-iterative framework that prioritizes high signal-to-noise contextual grounding over multi-round repair. DCAware integrates structured static context with selectively retrieved dynamic states, enabling precise and robust oracle generation without iterative feedback loops. Extensive experiments based on execution and mutation testing show that DCAware consistently improves fault-revealing effectiveness while maintaining high execution success, outperforming prior methods with substantially lower computational cost. Our results suggest that improving contextual quality is more effective than adding iterative repair complexity in the studied regression-oracle setting.

Article Search Artifacts Available Article: ase26main-p2670-p doi:10.1145/3832783.3834421

PSearch: Search-Based Patch Generation in the Era of LLM-Based Automated Program Repair
Haichuan Hu, Ye Shang, Weifeng Sun, and Quanjun Zhang
(Nanjing University of Science and Technology, China; Nanjing University, China; Singapore Management University, Singapore)
Large Language Models (LLMs) have substantially advanced Automated Program Repair (APR), yet most existing LLM-based APR methods still rely on trial-and-error to generate patches. Such a strategy explores candidate patches in a weakly structured manner, making it difficult to assess the future potential of search directions and allocate search budget effectively. To address this limitation, we propose Psearch, a search-based patch generation framework for LLM-based APR centered on iterative patch evaluation and refinement. Instead of treating patch generation as repeated independent sampling, Psearch maintains a structured search state over intermediate patches, continuously evaluates the promise of explored search paths, and prioritizes the most promising ones for further refinement. This design enables Psearch to abandon weak directions early and progressively approach correct fixes through long-horizon search. Importantly, Psearch can be integrated with different search algorithms, while our current implementation adopts Monte Carlo Tree Search as one effective instantiation. We evaluate Psearch on five widely used bug and vulnerability benchmarks. Experimental results show that Psearch correctly repairs 201 out of 835 bugs in Defects4J, outperforming all 12 state-of-the-art baselines. Psearch also fixes 27 of 79 vulnerabilities in VUL4J and resolves 164 of 300 issues in SWE-Bench-Lite. Moreover, with a patch size of 16, Psearch reduces monetary cost to roughly 50% of strong baselines while maintaining superior repair effectiveness. These results highlight the effectiveness of Psearch for improving LLM-based APR.

Article Search Article: ase26main-p2681-p doi:10.1145/3832783.3834422

Same Scrutiny, More Time: Eye Tracking Insights into Reviewing LLM-Labelled Code
Ranim Khojah, Francisco Gomes de Oliveira Neto, Mazen Mohamad, Julian Frattini, and Philipp Leitner
(Chalmers University of Technology, Sweden; University of Gothenburg, Sweden; RISE Research Institutes of Sweden, Sweden)
Modern software development increasingly involves the use of large language models (LLMs) to generate code. Despite their rapid advancement, LLMs remain prone to errors and hallucinations, emphasizing the importance of careful code inspection. However, in practice, developers' trust in LLM-generated code and their willingness to review it thoroughly may not align with these recommendations, which is a gap that remains largely unexplored. In this study, we conduct a Wizard-of-Oz experiment to examine how software engineers behave when code is explicitly labelled as LLM-generated during a code review task. We collect both behavioral data and participant feedback through eye-tracking and exit interviews. Combining Bayesian data analysis with qualitative analysis, we found that while the thoroughness of code review did not change for participants, they spent more time fixating on LLM-labelled code, indicating that the label itself influences attention. Practitioners also adapted their review strategy for LLM-labelled code by assessing the code based on specific criteria (e.g., logical correctness), or using the prompt to guide their review. These findings inform LLM-based tool design on labelling while incorporating the prompt as a software artifact. Our study reveals a gap between reviewers' intentions and actual reviewing behaviour, highlighting the need for software companies to revisit their AI policies (particularly regarding LLM-assisted development) to better support developers in reviewing LLM-generated code.

Article Search Artifacts Available Article: ase26main-p2693-p doi:10.1145/3832783.3834423

Breaking Customized LLMs for Coding: Automated Red Teaming for Instruction Backdoor Attacks
Yuchen Chen, Wei Cheng, Yuan Xiao, Weisong Sun, Chunrong Fang, Yang Liu, Zhenyu Chen, and Baowen Xu
(Nanjing University, China; Nanjing University of Aeronautics and Astronautics, China; Nanyang Technological University, Singapore)
LLM customization platforms allow users to build task-specific models for code intelligence tasks by embedding instructions into system prompts, without modifying the underlying model parameters. While these platforms lower the barrier to developing customized LLMs, they also introduce a new attack surface: instruction backdoor attacks, in which adversaries implant hidden malicious behaviors into customized instructions. However, existing attacks suffer from two key limitations. First, they often rely on explicit trigger patterns readily detected by platform-side or user-side inspection. Second, they require substantial manual effort to craft task-specific backdoored instructions, limiting their scalability.
In this paper, we propose ARIA, an automated red-teaming framework for crafting covert and effective backdoored instructions against customized LLMs. ARIA leverages an attacker LLM to iteratively generate and refine backdoored instructions, guided by structured feedback from the target LLM along three dimensions: stealthiness, clean-task utility, and backdoor effectiveness. We evaluate ARIA on three code intelligence tasks, using four representative LLMs, and compare it with three baseline attacks. Experimental results show that ARIA achieves the highest attack success rate of 0.945, while maintaining the best clean-task utility across all tasks. ARIA also generalizes well across programming languages and remains robust to generation temperature. Furthermore, ARIA significantly outperforms existing attacks in evading platform-side and user-side detection, achieving a false negative rate of up to 1.000, and stays effective against existing defense methods, demonstrating its strong generalizability and robustness.

Article Search Article: ase26main-p2706-p doi:10.1145/3832783.3834424

On the Reliability of Code Comprehension Proxies
Erfan Arvan, Nadeeshan De Silva, Oscar Chaparro, and Martin Kellogg
(New Jersey Institute of Technology, USA; William & Mary, USA)
Prior work on code comprehension uses different comprehension proxies—for example, Likert-scale ratings or answers to input-output questions about program snippets, usually collected from students, to approximate whether code is comprehensible to software engineers, but the relative reliability of these proxies is not known. This paper investigates the relative reliability of a collection of proxies common in the extant literature with a pair of human studies. First, we conducted an expert-consensus study with a panel of five professional software engineers to establish a ground-truth comprehensibility ranking of eight code snippets by adapting the Delphi expert-consensus protocol. The Delphi protocol is widely used for expert consensus under conditions of uncertainty in other domains such as medicine and national-security forecasting, but to our knowledge, this is its first application to code comprehension research. Second, we conducted a study with 44 student participants who completed comprehension tasks, allowing us to measure 14 comprehension proxies derived from the literature on the same set of eight code snippets. Finally, we conducted a correlation analysis on the results, concluding that proxies 1) derived from input-output questions and 2) that measure response time rather than accuracy are especially reliable. We also found that proxies derived from questions about program syntax (rather than semantics) are especially unreliable, regardless of measurement strategy, which draws into question the reliability of parts of the existing comprehensibility literature.

Article Search Artifacts Available Article: ase26main-p2877-p doi:10.1145/3832783.3834425

Discovering Performance Archetypes: Critical-Path-Aware Pattern Analysis and Regression Detection
Kaveh Shahedi, Heng Li, Maxime Lamothe, and Foutse Khomh
(Polytechnique Montréal, Canada)
Software performance analysis and prediction requires integrating multiple signals, as code structure alone cannot capture runtime behavior shaped by execution frequency, resource contention, and I/O patterns. We present a critical-path-aware performance analysis methodology that automatically discovers recurring performance patterns by synthesizing static code features, dynamic execution traces, and kernel-level resource data. In a preliminary study across six real-world C/C++ applications (SQLite, OpenSSL, Zstandard, FFmpeg, cURL, and jq), we first empirically confirm that static complexity metrics explain only 10.4% of the variance (ρ²) in critical path execution time, quantifying a gap that, while theoretically expected, had not been measured systematically across applications. Motivated by this finding, we analyze nearly 80,000 critical execution paths and address two research questions. First, we discover 13 distinct performance archetypes: recurring behavioral patterns that appear consistently across different applications, independent of their domain or implementation. Five of these patterns are near-universal and appear in at least five of the six applications studied. Notably, three of these archetypes are present in all six applications, and together, these common patterns account for 56.4% of all observed paths. Each archetype maps to specific resource profiles and optimization strategies that transfer across domains. Second, we leverage these archetypes within a multi-signal regression detection framework that triangulates path structure, resource consumption, and archetype deviations, achieving an F1-score of 0.867 and a 60.4% improvement over resource-only methods.

Article Search Article: ase26main-p2896-p doi:10.1145/3832783.3834426

Barista: Synthesizing Typestate Specifications with LLM Agents
Catarina Gamboa, Paulo Canelas, Ricardo Costa, Márcio Caetano, Jonathan Aldrich, and Alcides Fonseca
(Carnegie Mellon University, USA; University of Lisbon, Portugal)
Object-oriented classes encode implicit protocols that clients must follow, such as connecting a socket before sending data or opening a stream before reading. Mainstream programming languages like Java cannot validate the correct use of these protocols, but more advanced type systems, such as typestate systems, can detect these violations at compile time. However, these systems require a formalization of these implicit protocols. To overcome this limitation, we propose to extract object protocols from documentation using a combination of LLMs and test generation. We developed Barista, an agentic workflow that transforms class documentation into verifiable LiquidJava specifications through DFA-guided protocol extraction, test synthesis, and verification-driven refinement. We evaluated our approach on 35 Java classes with expert-written ground truth, achieving 87.0% precision and 68.3% recall, with 75% of specifications usable directly or with minor edits. Barista bootstraps these specifications for well-documented libraries, reducing the manual annotation burden and enabling compile-time protocol safety without specialized expertise.

Article Search Article: ase26main-p2950-p doi:10.1145/3832783.3834427

A Longitudinal Study of Android Apps Signing Key Protection
Mark Huasong Meng, Qing Zhang, Weirao Lu, and Chunyang Chen
(University College Dublin, Ireland; JD Group, China; TU Munich, Germany)
Android app signing relies on developer-managed credentials, making secure key protection essential for the integrity of the software supply chain. A recent platform key leakage incident involving two major OEM manufacturers demonstrates that even robustly designed signing mechanisms can be compromised due to developers' oversight. In this work, we conduct a longitudinal ecosystem study to characterize this threat by mining public repositories for Android signing credentials, recovering compromised keys via exposed passwords, and matching them against signatures from over 4,000 apps collected from major stores and OEM system images. Our analysis identifies 5,673 compromised keystores on GitHub and 26 unique certificates linked to 278 real-world apps. These include 26 third-party apps in public app stores and 252 preinstalled apps from seven manufacturers, collectively affecting over 10 billion users. We demonstrate the practical exploitability of these leaks through a proof-of-concept app replacement attack and identify spillover risks in non-smartphone platforms, including a popular automotive head-unit platform installed in over 1,100 vehicle models. Our results reveal that signing-key mismanagement is a systemic risk, underscoring the need for more rigorous key-management support in Android release engineering and distribution infrastructures.

Article Search Article: ase26main-p3058-p doi:10.1145/3832783.3834428

Clarity: From Formal System Designs to Verified RL Controllers
Austin O'Quinn, Conor Snedeker, Max Taylor, Lance Joneckis, and Christopher Stewart
(Ohio State University, USA; Boise State University, USA; Idaho National Lab, USA)
Safety-critical cyber-physical systems increasingly use reinforcement learning-based controllers, yet existing engineering methodologies cannot assure these systems’ safety. Assuring safety requires connecting three concerns that are currently addressed in isolation: the system’s design, the reinforcement learning pipeline, and runtime verification. This paper introduces CLARITY, a system that ingests designs written in SysML v2 and, through novel translational semantics to the nuXmv model checker, verifies safety requirements, synthesizes a reinforcement learning pipeline, and generates a runtime monitor. CLARITY enables fully traceable safety assurance arguments for reinforcement learning-based systems, closing the gap between system design, controller synthesis, and runtime assurance. We validate CLARITY on three CPS benchmarks of increasing complexity, demonstrating verified safety requirements, effective learned controllers, and lightweight runtime monitoring from a single SysML v2 specification.

Article Search Artifacts Available Article: ase26main-p3070-p doi:10.1145/3832783.3834429

Towards Verifiable Fairness Testing: Auditable Coverage over Data-Supported Input Domains
Yingqian Guo, Wentian Zhao, and Tian Song
(Beijing Institute of Technology, China)
Black-box models are increasingly deployed in high-stakes domains such as education, personnel hiring, and credit assessment, making systematic and auditable fairness testing particularly important. In such settings, black-box fairness testing is typically modeled as a search problem aiming to discover biases in the target model as much as possible. However, existing methods offer limited support for verifiable evaluation on a clearly stated, data-supported test domain: which reasonable regions have been tested, which have not, and whether exploration is balanced across the test domain remain unclear.
This paper redefines black-box fairness testing as an auditable process on a data-supported, reasonable input domain. We first construct a reasonable region from the reference data and divide it into a finite set of local coverage units. Then, we introduce a coverage scheduler that distributes testing work among the units and explicitly records tested and untested regions, thus achieving reproducible and verifiable sufficient testing. To improve discriminative discovery capabilities without compromising the coverage foundation, we further design a structure-aware intra-unit search mechanism that combines a structure-aware discriminative field (SDF) and directional influence refinement (DIR).
Experiments on multiple real-world tabular datasets show that our method enables auditable unit-level coverage accounting and achieves more balanced coverage, higher sample naturalness, and higher discrimination discovery effectiveness than representative black-box fairness testing baselines. Moreover, the discovered instances are effective for mitigating individual fairness violations through retraining.

Article Search Artifacts Available Article: ase26main-p3140-p doi:10.1145/3832783.3834430

PL4SA: Optimized Partial Library Selection for Efficient Static Analysis
Guohao Feng, Yifei Lu, and Minxue Pan
(Nanjing University, China)
With the rapid advancements in modern software engineering practices, software applications have become increasingly dependent on external libraries, making the choice between whole-program and application-only analysis more critical: the former often leads to timeouts due to high computational overhead, while the latter risks missing library-induced bugs. Recent research has explored a compromise path, i.e., partial inclusion of libraries to balance analysis efficiency and recall. However, achieving a more precise relevance analysis that enables a more balanced and accurate library selection while minimizing the associated overhead remains a significant challenge. This paper proposes PL4SA, a novel partial library selection approach that establishes precise relevance between client analyses and library methods using fine-grained whole-program points-to information. The key point of PL4SA lies in integrating points-to analysis with the partial library selection process in a progressively expanding manner. During this process, pointers are ranked according to their relevance to the client analysis; in each iteration, the most relevant pointer as well as its corresponding library is selected, while pointers dependent on the selected pointer are further expanded to reveal their points-to information. To support this analysis, extended pointers, a pointer-based abstraction that summarizes additional information flows into/out of method invocations and array accesses, serve as anchors for the progressive expansion of points-to information. Furthermore, PL4SA organizes this analysis in three steps, leveraging summarization techniques to enable the reuse of time-critical intra-procedural points-to analysis results within libraries across different projects, due to the inherent stability of library code. Collectively, these design choices contribute to PL4SA’s high efficiency. Our experimental results demonstrate that by including only 12% and 17% library methods, PL4SA achieves 24.6x, 9.0x speedup, and 99.0% recall, and both 100% precision for variable-type analysis and null-pointer analysis compared to whole-program analysis, significantly outperforming existing baselines.

Article Search Article: ase26main-p3256-p doi:10.1145/3832783.3834431

VeriSBOM: Secure and Verifiable SBOM Sharing via Zero-Knowledge Proofs
Gianpietro Castiglione, Shahriar Ebrahimi, and Narges Khakpour
(Newcastle University, UK)
SBOMs often contain sensitive information that organizations are unwilling to disclose in full. To avoid disruption of privacy and confidentiality in delivering plaintext SBOMs, we present VeriSBOM, a trustless SBOM framework that provides cryptographic verifiability of SBOMs using zero-knowledge proofs, i.e., third-parties can validate if software contains authentic dependencies distributed by official package managers and that the same dependencies are compliant with policy constraints. The verification process requires no trust in the SBOM publisher beyond the soundness of the underlying primitives, and third parties can independently check proofs against the public cryptographic commitments. We design and implement VeriSBOM, analyze its security, evaluate its performance on real-world package registries and SBOMs, and evaluate its usability.

Article Search Artifacts Available Article: ase26main-p3290-p doi:10.1145/3832783.3834432

Bifrost: Empowering Pretrained Language Model with Fallibility Representation for Log-Based Fault Diagnosis
Minghua He, Tong Jia, Lingzhe Zhang, Chiming Duan, Xinlong Zhao, Leyi Pan, Cheng Wang, Kangjin Wang, Yinghao Yu, Liping Zhang, Yifan Wu, and Ying Li
(Peking University, China; Tsinghua University, China; Alibaba Group, China)
Log-based fault diagnosis is crucial for runtime debugging and maintenance. Existing fault diagnosis methods use language models pre-trained on natural language (PLMs) for log representation. However, system faults are reflected in the multi-level structure of system logs. PLMs pre-trained on natural language struggle to comprehensively capture multi-level fault information, failing to meet the requirements of fault diagnosis. We refer to this information as fallibility representations. To address this problem, we propose a novel log representation learning method, Bifrost. It draws inspiration from the log analysis experience of Site Reliability Engineers and meticulously designs strategies based on self-supervised contrastive learning to learn the fallibility representations of logs. Across three public systems and one industrial ML-as-a-Service system, the log representations produced by Bifrost outperform existing PLMs by average margins of 9.83% in F1 for anomaly detection, 18.28% in HR@k for root cause localization, and 20.88% in Macro-F1 for fault identification.

Article Search Article: ase26main-p3295-p doi:10.1145/3832783.3834433

All or Nothing: How Library Type Annotations Affect Client-Side Errors in Python
Eric Asare, Luca Di Grazia, and Sarah Nadi
(New York University Abu Dhabi, United Arab Emirates; University of St. Gallen, Switzerland)
Python’s gradual typing promises that library-side type annotations will propagate safety benefits to downstream clients. Despite over 83% of popular libraries now containing annotations, no empirical evidence exists on whether these efforts actually reduce type errors in client code. We present the first large-scale empirical study of this downstream impact, analyzing the interaction between 13,751 actively maintained client repositories and over 4,178 distinct third-party libraries using the Pyright static type checker. Our results show that (i) 70% of client repositories contain at least one type error stemming from third-party API usage, (ii) the error distribution is heavily skewed, with a small subset of widely used libraries, i.e., numpy, django, pandas, and pytorch, accounting for a disproportionate share of the ecosystem-wide error burden, (iii) measurable benefits of reduced error risks and propagation can only be seen with near-complete annotation coverage (≥ 90%), and (iv) although 60% of library-client pairs are aligned in their typing efforts, only 10% achieve the ideal scenario where highly annotated libraries meet client-side static type checking. Our findings expose a pervasive ecosystem gap: client adoption of static type checking outpaces library annotation efforts, with 19% of cases representing missed opportunities where clients use type checkers but their dependencies lack sufficient annotations. These results call for prioritizing comprehensive annotation of widely used libraries to unlock the full potential of Python’s gradual typing.

Article Search Artifacts Available Article: ase26main-p3300-p doi:10.1145/3832783.3834434

Tensor-Based Batch Fuzzing with Adaptive Perturbation Scaling for Deep Neural Networks
Guanqin Zhang and Yulei Sui
(University of New South Wales, Australia)
Deep neural networks (DNNs) are increasingly deployed in safety-critical domains such as autonomous driving and medical diagnosis, yet their opaque, high-dimensional parameter spaces make it difficult to systematically assess model reliability on unseen inputs. Existing coverage-guided sequential fuzzing frameworks for DNN inherit a one-input-per-iteration design from traditional software fuzzing and apply uniform perturbation budgets across all input dimensions, limiting both testing throughput (i.e., inputs processed per unit time) and the precision of input-space exploration.
We present a new specification-aware batch fuzzing framework with adaptive perturbation scaling that addresses both limitations. Rather than relying on a fixed global perturbation radius є, our approach derives mutation step sizes from specification-defined feasible ranges (i.e., the gap between lower and upper bounds) using a shared scale factor. This scaling can be applied either as a global scalar (isotropic) or as per-dimension step sizes (anisotropic), enabling perturbations to remain consistent with the underlying constraint structure. As a result, the fuzzer can explore input spaces with heterogeneous feature scales more effectively across all specifications in the batch. We embed input constraints and output property checks directly into the network as non-trainable layers, yielding a wrapped model that processes B specification instances in a single batched iteration, which substantially improves fuzzing efficiency and exploration of counterexamples.
We evaluate our framework extensively on three benchmarks, covering six networks and over 400 specifications across TrafficSigns, Cifar100, and TinyImageNet. Our tensor-based fuzzing achieves up to 40× higher throughput and 4× more violations than the sequential baseline under the same time budget, demonstrating significantly improved effectiveness in specification-guided fuzzing.

Article Search Artifacts Available Article: ase26main-p3318-p doi:10.1145/3832783.3834435

ChangeWiki: Repository-Level Change Summarization Augmented with Architecture Semantics
Zijun Wang, Wuxia Jin, Jiale Li, Zhanhong Mou, and Ting Liu
(Xi'an Jiaotong University, China)
Understanding software evolution at the repository level is essential for tasks such as release management, code review, and regression testing. However, existing change summarization techniques are largely confined to single commits or functions, failing to capture cross-commit evolutionary context, while recent repository-level understanding tools focus on static documentation of single-version codebases, overlooking evolution over time. To bridge this gap, we propose ChangeWiki, an architecture semantic-augmented approach for repository-level change summarization. ChangeWiki first constructs hierarchical, multi-granularity change entity trees enriched with multi-source contexts to infer change intent and impact. It then recovers modular structures across versions and employs a bidirectional context-guided aggregation mechanism, synthesizing code-level changes from the bottom up while injecting architectural semantics as top-down constraints. ChangeWiki finally generates coherent, traceable summaries that capture both local modifications and system-level evolution. We construct comprehensive benchmarks and evaluate ChangeWiki on real-world projects. Experimental results demonstrate significant improvements in semantic accuracy, completeness, and cross-granularity traceability. A prototype of ChangeWiki has been deployed in an industrial setting for upstream-downstream code evolution, where developers reported that the hierarchical summaries significantly reduced manual effort in issue localization caused by upstream changes and show potential for automating version release notes.

Article Search Article: ase26main-p3335-p doi:10.1145/3832783.3834436

Refine2Diff: Detecting Protocol Specification–Implementation Inconsistencies via Specification-Driven Code Refinement
Yuekun Wang, Lili Quan, and Xiaofei Xie
(Singapore Management University, Singapore)
Detecting specification–implementation inconsistencies is critical and fundamentally challenging due to the semantic gap between natural-language specifications and executable code. Existing approaches often require substantial manual effort to translate specification requirements into formal properties for verification or testing. While recent LLM-based methods have shown promise in automating this process, they suffer from hallucination and unreliable reasoning, producing false positives. To address these limitations, we present Refine2Diff, a novel framework for automatically detecting and validating specification–implementation inconsistencies in network protocols. Rather than relying solely on a direct binary judgment from an LLM, we use the specification as an instruction to refine the implementation, thereby exposing inconsistencies through observable behavioral differences between the original and refined implementations. Specifically, Refine2Diff performs context-aware refinement to construct a reference implementation and supports refinement-guided PoC generation by deriving preconditions that trigger behavioral differences and satisfying them through specification-guided mutation. We evaluate Refine2Diff on 10 real-world implementations spanning six network protocols. For inconsistency detection, Refine2Diff achieves a precision of 0.84 (1.4× higher than the strongest baseline), improving further to 0.98 after the PoC-based confirmation. For PoC generation, Refine2Diff achieves an 82% success rate, outperforming all baselines by 3×–9× while requiring up to 5.7× fewer executions. In total, Refine2Diff uncovers 30 previously unknown inconsistency bugs, 22 confirmed by developers.

Article Search Article: ase26main-p3339-p doi:10.1145/3832783.3834437

Free Is More: Empowering Software Agents via Issue-Free Trajectory Learning and Entropy-Aware RLVR Training
Xin-Cheng Wen, Binbin Chen, Haoxuan Lan, Hang Yu, Peng Di, and Cuiyun Gao
(Harbin Institute of Technology, Shenzhen, China; Ant Group, China; Zhejiang University, China; University of New South Wales, Australia)
Large language models (LLMs) have transformed the software engineering landscape. Recently, numerous LLM-based agents have been developed to address real-world software issue fixing tasks. These agents are typically equipped with various coding tools, enabling them to autonomously generate action trajectories for end-to-end problem-solving. Despite their state-of-the-art performance, these agents still face two primary challenges: (1) Lack of sufficiently high-quality issue descriptions: Inconsistencies between issue descriptions and their corresponding solutions introduce noise and ambiguity that can mislead automated agents, restricting their ability to derive effective solutions. Furthermore, high-quality issue-PR pairs are difficult to acquire at scale, and the available datasets remain relatively limited in size. (2) Uncertainty introduced by bash tools prevents stable training and sufficient exploration: Bash tools serve as the primary interface for code execution and file manipulation in SWE agents, making them integral to virtually all agent operations. However, bash tools introduce substantial uncertainty into the agent's decision-making process. In reinforcement learning with verifiable rewards (RLVR), the fixed probability ratio clipping mechanism exhibits limited adaptability to such varying uncertainty states. Specifically, when bash tool invocations trigger high-entropy states, excessively restrictive clipping constraints impede both exploration and stable learning, hindering the training efficiency.
To address these challenges, we propose SWE-Fuse, an issue-description-aware training framework that fuses issue-description-guided and issue-free samples for training SWE agents. It consists of two key modules: (1) An issue-free-driven trajectory learning module, which comprises: a multi-step trajectory construction component for generating high-quality multi-turn reasoning-action trajectories, a trajectory data filtering process for ensuring data quality, and issue-free-driven supervised fine-tuning to mitigate potentially misleading issue descriptions while enabling the model to learn step-by-step debugging processes; and (2) An entropy-aware RLVR training module, which adaptively adjusts training dynamics through entropy-driven clipping. It applies relaxed clipping under high entropy to encourage exploration, and stricter clipping under low entropy to ensure training stability. We evaluate SWE-Fuse on the widely studied SWE-bench Verified benchmark shows to demonstrate its effectiveness in solving real-world software problems. Specifically, SWE-Fuse outperforms the best 8B and 32B baselines by 41.9% and 59.3% in solve rate, respectively. Furthermore, integrating SWE-Fuse with test-time scaling (TTS) enables further performance improvements, achieving solve rates of 49.8% and 65.2% under TTS@8 for the 8B and 32B models, respectively.

Article Search Article: ase26main-p3466-p doi:10.1145/3832783.3834438

How Quantum Bugs Live and Die: A Lifecycle-Based Empirical Study of Bugs in Quantum Software
Yasai Shi, Xiangxin Meng, Xiangjie Huang, Jian Zhang, Tianyu Wo, and Xu Wang
(Beihang University, China)
Despite growing evidence that bugs in quantum software exhibit distinctive characteristics, existing studies are typically scoped to individual frameworks or single quality attributes and do not examine how quantum-specific bugs behave across the broader software lifecycle. To address this gap, we construct QBugsSet, a lifecycle-based benchmark containing 2,429 bugs and 1,793 realistic negative samples, mined from real GitHub bug-fixing commits in the Qiskit, Cirq, and PennyLane ecosystems, and use it to conduct a comprehensive empirical study comparing quantum-specific and classical bugs across lifecycle distribution, bug-fix latency and pre-fix spatial propagation, repair burden, and LLM-based debugging effectiveness. We find that quantum-specific bugs are more concentrated in semantically critical stages, tend to persist longer before repair (median 17 vs. 7 days), and more often require coordinated multi-hunk repairs (37.8% vs. 27.3%). These lifecycle characteristics highlight a practical limitation of current automation: even the best frontier LLM achieves an end-to-end full repair rate of only 3.82% on quantum-specific bugs, compared with 8.51% on classical bugs. Overall, these findings provide evidence-based guidance for future debugging support that integrates stronger quantum-aware semantic reasoning with broader repository context.

Article Search Article: ase26main-p3477-p doi:10.1145/3832783.3834439

Automated Table Reproduction via Code Generation
Doehyun Baek and Michael Pradel
(CISPA Helmholtz Center for Information Security, Germany)
Reproducibility is an important goal in computer science research, e.g., for artifact evaluation and to build upon experimental results of prior work. Recently, LLM agents are being used to automatically reproduce research results, but they fail to provide executable evidence of reproduction and do not consider the method of reproduction, which limits their usefulness. We present Artisan, an LLM agent that reproduces tables of numeric results, given a paper and its artifact. The approach is enabled by two key contributions: First, we frame the reproduction problem as a code generation task, enabling users to audit and re-run the resulting reproduction script independently of the agent. Second, we design automated judging mechanisms that steer the agent toward correct results without exposing them, while preventing shortcuts like copying precomputed results. To evaluate Artisan, we introduce Artisan-Bench, the first benchmark assessing the ability to generate code that reproduces research results. Artisan-Bench comprises 60 tasks derived from 23 software engineering papers. Our experiments show that Artisan is effective and efficient, with the added benefit of aiding the discovery of 20 new errors in either the paper or artifact.

Article Search Artifacts Available Article: ase26main-p3621-p doi:10.1145/3832783.3834440

MicroAgent: Context-Augmented Multi-agent Framework for Automatic Microservice Decomposition
Zishan Su, Junjie Huang, Shiwen Shan, Xingyan Chen, Hui Zeng, Yuxin Su, Yanlin Wang, and Michael R. Lyu
(Chinese University of Hong Kong, China; Sun Yat-sen University, China)
The adoption of Microservice Architecture (MSA) has revolutionized software engineering by enhancing scalability, agility, and maintainability over traditional monolithic applications. As more developers transition their legacy systems to microservice-based architectures, effective microservice decomposition—partitioning monolithic applications into highly cohesive services—becomes vital. However, this decomposition task presents significant challenges. Manual approaches are time-consuming and labor-intensive. Existing automated methods often fail to capture the necessary semantic insights from complex applications, while naive applications of Large Language Models tend to overlook crucial contextual information and design principles, leading to suboptimal results.
To address these challenges, we propose MicroAgent, a Context-Augmented Multi-Agent Framework for Microservice Decomposition. Our framework divides the decomposition process into five distinct subtasks and assigns each to a specialized agent. To enhance the effectiveness of each agent, we provide tailored, multi-granularity context that keeps its analysis focused and mitigates information overload. Furthermore, to ensure the decomposition adheres to established design principles, we integrate analytical tools that guide the agents’ decision-making. Experimental evaluations on 10 Java Web applications demonstrate that MicroAgent achieves an average decomposition accuracy of 89.2%, outperforming the state-of-the-art method by 24.6%. We also conduct a case study to highlight the practical benefits of our design.

Article Search Artifacts Available Article: ase26main-p3640-p doi:10.1145/3832783.3834441

Optimization-Free Repair of Code LLMs: A Closed-Form Fisher-Guided Bayesian Approach
Zhiyu Duan, Chong Bian, Yizhuo Zhang, and Shunkun Yang
(Beihang University, China; China Mobile, China)
Large language models (LLMs) are increasingly used in software engineering, yet their code generations remain vulnerable to subtle functional errors, making reliable post-hoc repair essential. Existing repair methods typically apply localized parameter updates without explicitly accounting for the heterogeneous importance of pre-trained knowledge, which can entangle targeted correction with unintended changes to general coding ability. We propose Fisher-Informed Semantic Repair (FISR), a principled framework that treats code LLM repair as a Bayesian posterior update constrained by pre-trained prior knowledge. It estimates parameter-wise semantic importance through the Fisher Information Matrix, using local curvature information to identify high-confidence directions that should be preserved during repair. It then introduces Fisher-weighted structural constraints to guide corrective updates away from knowledge-critical parameters and toward more adaptable regions of the parameter space. To make this process scalable, we combine structured curvature approximation with low-rank precision decomposition, enabling efficient closed-form updates within a safe repair subspace rather than relying on iterative retraining. Extensive experiments on three backbone code models and six software engineering benchmarks show that FISR achieves the best overall trade-off among repair accuracy, stability, and efficiency. It reaches up to 95.8% Exact Match, improving Exact Match by up to 3.1% and Pass@1 by up to 3.4% over strong prior repair methods. It also demonstrates stronger held-out stability while substantially reducing repair cost, requiring only 40%-67% of the end-to-end repair time and 14%-20% of the update-time cost of competitive editing-based baselines. These results demonstrate that the proposed method enables accurate, safe, and efficient semantic repair for code LLMs.

Article Search Article: ase26main-p3860-p doi:10.1145/3832783.3834442

Pattern over Pixels: Measuring Pattern Completion Bias in Multimodal Code Generation
Khai-Nguyen Nguyen, Oscar Chaparro, and Antonio Mastropaolo
(William & Mary, USA)
Multimodal large language models (MLLMs) are increasingly used to translate webpage screenshots into front-end code, but repeated UI patterns may sway them toward visually incorrect yet pattern-consistent outputs. In this work, we test how repeated webpage patterns hurt MLLM accuracy on an objective screenshot-to-code fill-in-the-blank task. We introduce the first benchmark for visual pattern-completion bias, where one localized element in a repeated UI pattern is perturbed and the model must recover the masked width or font-size value from the screenshot and HTML context. Starting from 30 webpages curated from the Design2Code dataset, we build 1,440 evaluated screenshots spanning structural card and text-style patterns under standard and noise-overlaid conditions. We evaluate five frontier MLLMs and find that all are strongly biased toward the repeated baseline. Mean bias rate reaches 69.78% on card-width perturbations and 80.22% on text font-size perturbations, while mean accuracy is only 21.17% and 7.89%, respectively.  performs best but still drops from 68.61% accuracy on cards to 13.89% on text, while  reaches 96.11% bias on text. Noise, subtler perturbations, and boundary positions further increase bias rate. Reasoning analysis further shows that greater reasoning effort correlates with lower bias, yet qualitative evidence reveals that models can identify the anomalous element and still override it with the pattern-consistent answer. Our results identify a concrete failure mode in multimodal code generation and show that its severity is strongly associated with visual saliency.

Article Search Article: ase26main-p3922-p doi:10.1145/3832783.3834443

GPU-Accelerated Belief Propagation for Program Analysis
Haoyu Feng and Xin Zhang
(Peking University, China)
Belief Propagation (BP) is a widely used approximate inference algorithm in probabilistic graphical models (PGMs), but it is computationally expensive when applied to large-scale program analysis. Existing GPU-based approaches are unable to support flexible update strategies and have yet to integrate logical constraints with GPU acceleration, leading to challenges in both generality and efficiency.
We present FastLBP, a GPU-accelerated BP framework for program analysis. We propose a unified representation for specifying flexible update strategies required in program analysis, along with a dependency analysis algorithm to enable parallel execution. Furthermore, we implement BP with local structures on GPUs by assigning individual threads to message computations and utilizing a memory-efficient representation. Experiments on SmartFL and Bingo show that FastLBP achieves average speedups of 17.42× and 2.82× over CPU-based approaches on SmartFL and Bingo, respectively, and 6.14× over the GPU-based approach on SmartFL, while preserving accuracy. Moreover, FastLBP supports update strategies that existing GPU-based approaches cannot support, demonstrating its improved generality for real-world program analysis.

Article Search Artifacts Available Article: ase26main-p4025-p doi:10.1145/3832783.3834444

Inferring the Shape of Data Frames in R Programs using Abstract Interpretation
Oliver Gerstl, Florian Sihler, and Matthias Tichy
(Ulm University, Germany)
Data frames are a fundamental data structure in many data analysis tasks and are widely used in programming languages like R. Due to their omnipresence in data analysis, there are many functions that operate on their shape and content, for example, to clean and transform study data. However, languages like R do not offer static guarantees on data frames making it difficult to reason about their shape at a specific point in the program.
In this paper, we present a novel static analysis to infer the shape of data frames in R programs using abstract interpretation by tracking the ensured and potential column names, as well as the potential number of columns and rows. For this, we use a reduced product domain and define abstract semantics for the most commonly used data frame operations, such as mutating, filtering, and subsetting. We evaluate the correctness and accuracy of our analysis on a selection of 78 executable real-world R scripts achieving empirical evidence for soundness by never under-approximating the data frame shape. Additionally, we demonstrate the ability of our analysis to infer the shape of data frames on a large dataset of 33,314 real-world R scripts by inferring concrete shape constraints for 42.1 % and exact shapes for 0.9 % of the data frame operations, improving to 58.7 % and 4.2 % if all datasets read in these scripts are available to our analysis. Using the inferred data frame shapes, we identified 40 real-world R scripts containing potential invalid data frame accesses. This shows the potential of our analysis to significantly support researchers in using data frames in data analysis.

Article Search Article: ase26main-p4457-p doi:10.1145/3832783.3834445

Translation Tag Team: Formal Rules and LLMs Translate More Macros Together Than Apart
Brent Pappas, Joseph Zalusky, Zachary Burkett, and Paul Gazzillo
(University of Central Florida, USA)
Modern critical software infrastructure is largely written in C. Since C lacks memory safety, researchers are investigating automatic translation of C to safer languages like Rust. But real-world C software consists of more than just C code, often using named code fragments called macros which are not part of the C language proper. State-of-the-art techniques avoid translating macros by preprocessing C code first before translating it. But this approach produces translations that are dissimilar to the original C code, because preprocessing inlines all macro definitions. To preserve macro usage in translated code, we study the language features that macros and C share and distill them into the first formally- specified translator, MerC. To evaluate MerC, we introduce the first macro translation benchmark, MacroBench, with test cases based on macros randomly sampled from real-world C programs. We find that MerC supports 50% of MacroBench’s macro test cases. We also use MacroBench to evaluate how effective large language models (LLMs) are at performing the previously-unstudied task of macro translation. LLMs translate 22% to 77% more of MacroBench than MerC, but with 8% and 28% of these translations being incorrect translations requiring additional validation by developers. In contrast, MerC only produces correct translations. Our key insight is that running MerC first then using LLMs on the remainder reaps greater benefits than using either technique alone. This tag team approach has an average failure rate 32% lower than that of LLMs, while also translating an average of 51% more test cases than MerC.

Article Search Artifacts Available Article: ase26main-p5-p doi:10.1145/3832783.3837412

ProbGuard: Proactive Runtime Monitoring for LLM Agent Safety via Probabilistic Prediction
Haoyu Wang, Christopher M. Poskitt, Jiali Wei, and Jun Sun
(Singapore Management University, Singapore; Xi'an Jiaotong University, China)
Large Language Model (LLM) agents increasingly operate across domains such as robotics, virtual assistants, and web automation. However, their stochastic decision-making introduces safety risks that are difficult to anticipate during execution. Existing runtime monitoring frameworks, such as AgentSpec, primarily rely on reactive safety rules that detect violations only when unsafe behavior is imminent or has already occurred, limiting their ability to handle long-horizon dependencies. We present ProbGuard, a proactive runtime monitoring framework for LLM agents that anticipates safety violations through probabilistic risk prediction. ProbGuard abstracts agent executions into symbolic states and learns a Discrete-Time Markov Chain (DTMC) from execution traces to model behavioral dynamics. At runtime, the monitor estimates the probability that execution will remain safe from the current state, and triggers an intervention when this probability falls below a user-defined threshold. To improve robustness, ProbGuard incorporates semantic validity constraints in the abstraction and admits a PAC-style analysis that characterizes the sample complexity required to certify the learned model under standard assumptions. We evaluate ProbGuard in two safety-critical domains: autonomous driving and embodied household agents. Across evaluated scenarios, ProbGuard consistently predicts traffic law violations and collisions in advance, with warnings up to 15.84 seconds at a threshold yielding no false alarms, and up to 38.66 seconds at stricter thresholds. In embodied agent tasks, ProbGuard's re-prompting intervention mode reduces unsafe behavior by 65.37% relative to the unmonitored baseline while retaining 80.4% of the baseline task completion; a stricter halting configuration reduces unsafe behavior by 93.60% at a larger cost in completion. ProbGuard is implemented as an extensible open-source runtime monitor integrated with the LangChain agent framework and introduces minimal runtime overhead.

Article Search Article: ase26main-p53-p doi:10.1145/3832783.3837413

Tangent: An Empirical Study of Testing Practices for LLM-Based Agent Applications
Rangeet Pan, Tyler Stennett, Divya Sankar, Bridget McGinn, Alessandro Orso, Raju Pavuluri, Saurabh Sinha, and Maja Vukovic
(IBM Research, USA; Georgia Institute of Technology, USA; University of Georgia, USA)
Agents built on large language models (LLMs) are increasingly used to develop applications that perform complex, multi-step tasks involving reasoning, tool use, and interaction with external environments. Despite rapid advances in benchmarking LLM-based agents, little is known about how they are tested in practice. In particular, key aspects of testing---such as testing levels, objectives, data patterns, test complexity, and validation strategies---for agent applications remain underexplored. In this paper, we present an empirical study of testing practices for LLM-based agent applications using a corpus of mined open-source projects. We construct an extensive dataset of agent applications, tools, and tests, and manually label 2,572 test methods from 240 modules. From this analysis, we develop a taxonomy of 23 testing patterns across test fixtures, data, objectives, and assertions, and characterize tests by level (unit, module, integration). We complement this with structured interviews of 10 senior industry practitioners building agentic systems. Our results show that testing of LLM-based agent applications is dominated by narrowly scoped unit tests, with limited coverage of complex interactions, realistic scenarios, and non-functional requirements. Tests often rely on simplistic inputs, heavy mocking, and shallow validation, and exhibit low structural complexity. Although industry practice places greater emphasis on non-functional testing than open-source projects, both reveal common gaps, including the lack of formal testing foundations, unclear test objectives, and challenges in generating high-quality test data. Based on these findings, we identify research directions toward more systematic and rigorous testing of agent applications, including foundations for agent testability, formalized test objectives, and fault-based testing techniques tailored to agentic systems. We also release a curated dataset of testing artifacts to support future research.

Article Search Article: ase26main-p87-p doi:10.1145/3832783.3837414

A Unified Model for Cross-Domain Clone Detection via Model Merging
Palash R. Roy, Banani Roy, Kevin A. Schneider, and Chanchal K. Roy
(University of Saskatchewan, Canada)
The growing diversity of code clone types, from syntactic copies to cross-language semantic clones to AI-generated duplicates, has created a fragmentation crisis in clone detection. Current deep learning detectors are domain specialists that degrade significantly outside their training distribution. In our evaluation, F1 drops exceed 70% across domains. Deploying multiple specialized models is impractical, yet training a single cross-domain detector requires simultaneous access to all training data. To address this, we investigate model merging, a family of post-hoc techniques that operate solely on trained checkpoints. We systematically evaluate parameter merging with five task-vector methods (Task Arithmetic, TIES, DARE-TIES, WUDI, PCB), architecture merging via greedy layer stitching, and cross-tokenizer alignment. Our study spans four pre-trained code models, three benchmarks, and twelve merging configurations. Same-base TIES merging creates effective cross-domain detectors, validated across two model families and three random seeds. It reaches 0.865 combined F1 on UniXcoder, 93% of multi-task training performance without any training data at the merging step. WUDI achieves the highest in-distribution combined F1 at 0.899, but TIES generalizes better to unseen AI-generated clones. We therefore recommend TIES for the practical recipe. Cross-base merging yields only marginal and high-variance gains across all five methods. This indicates that task vector compatibility through a shared pre-trained base is the binding factor for effective merging in our setting. Merged encoder-based detectors also outperform zero-shot instruction-tuned code LLMs on GPTCloneBench (0.609 vs. 0.454 F1 for the strongest LLM baseline) at substantially lower per-pair inference cost. They also generalize up to 4× better than multi-task training to unseen AI-generated clones, suggesting a trade-off between in-domain performance and OOD robustness. This work provides one of the first systematic empirical studies of model merging for software engineering and a practical recipe for building cross-domain code clone detectors.

Article Search Artifacts Available Article: ase26main-p92-p doi:10.1145/3832783.3837415

Don’t Retest Everything: Auditable Regression Scoping for Safety-Critical Systems
Francesco Basciani, Daniele Masti, Patrizio Pelliccione, Alberto Petrucci, Franco Raimondi, Emerson Sales, and Paolo Serri
(Gran Sasso Science Institute, Italy; Gran Sasso Tech, Italy; Thales Alenia Space Italia, Italy)
In safety-critical software development, changes often span technical and organisational boundaries, e.g., repositories, interfaces, libraries, and teams. Under certification and traceability requirements, these changes create a difficult regression-testing problem: organisations must determine what must be revalidated and provide an auditable justification for those decisions. In practice, the resulting uncertainty often drives overly conservative strategies, where large portions of the system are retested after changes or updates, even when a narrower scope would suffice.
In this paper, we present CoCoS, an automated regression-testing approach for safety-critical systems written in C and developed under traceability and certification constraints. Developed end-to-end with Thales Alenia Space Italia, a large international aerospace company, and grounded in avionics practice, CoCoS addresses real certification-driven needs rather than an abstract setting. It combines semantic differencing, abstract syntax tree analysis, and function-level call-graph propagation to identify semantically relevant changes across repository and component boundaries, and to derive conservative regression scopes with human-readable justifications. CoCoS prioritizes conservativeness, explainability, and auditability, thereby supporting defensible regression-scoping decisions in regulated workflows. We evaluate CoCoS in an industrial setting with Thales Alenia Space Italia, using a realistic multi-repository aerospace software environment involving application code and external libraries. The results show that CoCoS supports the conservative bounding of regression scopes while preserving compliance-oriented reasoning and providing practical evidence for test planning in safety-critical environments.

Article Search Article: ase26main-p115-p doi:10.1145/3832783.3837416

Recovering Revisions of Pull Requests with Altered History
Gengyi Sun, Georges Aaron Randrianaina, Tao Xiao, Yasutaka Kamei, and Shane McIntosh
(University of Waterloo, Canada; Kyushu University, Japan)
In pull-based development, Pull Requests (PRs) undergo peer review. To address review feedback, authors may "force-push" after amending, squashing, or rebasing commits. Such revisions can conflate PR-specific changes with updates on the base branch, hindering further review and introducing noise into PR-based datasets.
In this paper, we empirically study 935,011 PRs from 1,445 well-maintained GitHub projects. We observe 568,218 force-push events in 242,624 PRs that span 1,366 projects. We denote the HEAD commits of the PR before and after force-pushing as a force-push pair, and observe that 210,837 pairs (37.1%) have a relationship that conflates PR changes with updates to the base branch.
To systematically isolate PR-specific changes in such revisions, we propose AntiDiffamine---an ancestry-based approach that aligns commits on a common reference. AntiDiffamine can automatically deconflate 98.1% of the studied revisions, decreasing the diff size by one to three orders of magnitude.
We also evaluate AntiDiffamine on fail-fix pairs, i.e., revisions that fix failing builds. A case study of Kubernetes reveals that 130 of 523 pairs have conflated diffs. Moreover, the fixing files identified when AntiDiffamine is applied overlap more with a human-labelled ground truth than when conflation is untreated or excluded.
AntiDiffamine is a first attempt at addressing force-push-based diff conflation, which is having a real-world impact on practical and research workflows. To ease practical adoption, we provide a template GitHub Action, which scans PR updates for force-push events and applies AntiDiffamine to deconflate the diff.

Article Search Artifacts Available Article: ase26main-p120-p doi:10.1145/3832783.3837417

Making Theft Useless: Adulteration-Based Protection of Proprietary Knowledge Graphs in GraphRAG Systems
Weijie Wang, Peizhuo Lv, Yan Wang, Weiqing Huang, Rujie Dai, Guokun Xu, Qiujian Lv, Hangcheng Liu, Hao Hu, and Jiaheng Zhang
(Institute of Information Engineering at Chinese Academy of Sciences, China; University of Chinese Academy of Sciences, China; Nanyang Technological University, Singapore; Henan Key Laboratory of Information Security, China; National University of Singapore, Singapore)
Graph Retrieval-Augmented Generation (GraphRAG) has emerged as a foundational architecture for modern intelligent software systems, enhancing Large Language Models (LLMs) with proprietary Knowledge Graphs (KGs) in knowledge-intensive and automated tasks. As these KGs often represent an organization's highly valuable intellectual property (IP), they face a significant risk of theft for private use. In this scenario, attackers operate in isolated environments. This private-use threat renders passive defenses like watermarking ineffective, as they require output access for detection. Simultaneously, the strict performance requirements of dependable software operations make strong cryptographic solutions, which incur prohibitive overhead, impractical. To address these challenges, we propose AURA, a novel framework based on Data Adulteration designed to make any stolen KG unusable to an adversary. Our framework pre-emptively injects plausible but false adulterants into the KG. For an attacker, these adulterants deteriorate the retrieved context and lead to factually incorrect responses. For authorized users, a secret key enables the efficient filtering of all adulterants via encrypted metadata tags before they are passed to the LLM, ensuring query results remain completely accurate. Our evaluation demonstrates the effectiveness of this approach: AURA degrades the performance of unauthorized systems to an accuracy of just 5.3%, while maintaining 100% fidelity for authorized users with negligible overhead. Furthermore, AURA proves robust against sanitization attempts, retaining 80.2% of adulterants.

Article Search Article: ase26main-p131-p doi:10.1145/3832783.3837418

Decompiling the Web: Automated Semantic Recovery of Post-compilation Abstraction Leaks
Shriyans Sudhi and Yinxi Liu
(Rochester Institute of Technology, USA)
Modern web applications rely on advanced compilation pipelines to transform high-level source code into heavily optimized execution bundles. This structural transformation introduces a systemic security gap that we formalize as Post-Compilation Semantic Collapse. During the build process, the application state is fractured into disjointed execution fragments, and dependencies are subsequently orchestrated via dynamic runtime engines. While developers operate under the assumption of server-client isolation, modern bundlers can weaken this boundary to optimize performance, silently migrating sensitive backend context and internal routing logic into public client-side artifacts. Traditional security scanners are poorly matched to these architectural exposures. By evaluating compiled JavaScript as a flat, static resource, existing tools fail to reconstruct dynamic dependency graphs or interpret the proprietary streaming protocols required to map the modern attack surface.
To overcome this structural gap, we introduce JS Recon, an automated semantic recovery framework that performs decompiler-like analysis for modern JavaScript bundles. Because bundlers obscure execution paths using cryptographic hashes and functionally transformed arrays, pure static parsing is insufficient. JS Recon defeats this obfuscation by coupling static structural analysis with targeted dynamic execution. Specifically, it statically lifts heavily minified payloads into an Abstract Syntax Tree (AST) to isolate obfuscated routing logic, and subsequently evaluates these localized runtime dictionaries within a dynamic native sandbox. Furthermore, to resolve decentralized server-side dependencies, the pipeline actively manipulates proprietary React Server Component (RSC) streaming protocols, forcing the backend to dynamically serialize and expose routing state not present in static artifacts.
We deployed JS Recon against the Tranco Top 1M domains to execute a large-scale measurement study. Our pipeline extracted over 2.5 million orphaned sourcemaps, achieving a 97.04% extraction rate in controlled environments compared to 0% for baseline tools. Furthermore, JS Recon recovered 6,210 verified secret instances, establishing a 6.6x discovery multiplier over the leading reconnaissance spider. Our measurement identifies a widespread ecosystem issue, finding 447 globally unique verified secrets distributed across 379 root domains. This study shows that automated semantic recovery is needed to audit build artifacts and map the obscured attack surfaces of modern web deployments.

Article Search Artifacts Available Article: ase26main-p145-p doi:10.1145/3832783.3837419

AutoSQL: Extracting SQL Templates from Imperative ORM Code in Large-Scale Repositories
Junsong Pu, Yichen Li, Zhuangbin Chen, Zhihan Jiang, and Zibin Zheng
(Sun Yat-sen University, Zhuhai, China; Chinese University of Hong Kong, Hong Kong)
Suboptimal SQL queries can significantly degrade the performance of cloud systems, motivating the extraction and auditing of SQL statements before deployment. However, Go ORM frameworks construct SQL imperatively through scattered method-call sequences, making it difficult to statically recover the resulting SQL templates. We present AutoSQL, a system that reconstructs SQL templates from Go ORM code. AutoSQL constructs a Code Index, a directed graph that captures structural dependencies between functions, types, and global variables as navigable edges. It then traces upstream call chains from ORM invocation sites to identify database-interacting functions as entry points. For each entry point, an LLM agent traverses the Code Index to collect code slices that influence SQL generation, switching to pattern-based search when the graph cannot resolve a retrieval goal. We call this strategy Hybrid Context Retrieval. Once sufficient context is collected, the agent synthesizes SQL templates. Evaluation on a benchmark of 579 test-covered entry points and 1,186 runtime-traced SQL statements from five large-scale Go repositories shows that AutoSQL achieves 68.04% to 72.18% recall, exceeding the static reachability baseline by 11.80% to 15.94% and outperforming existing methods by 8.52% to 48.57%.

Article Search Article: ase26main-p169-p doi:10.1145/3832783.3837420

SIMD-Accelerated Sparse Bit-Vectors for Pointer Analysis
Zhaoyang Tan, Peisen Yao, and Kui Ren
(Zhejiang University, China)
Pointer analysis is a core component of program-analysis frameworks. Inclusion-based solvers repeatedly update points-to sets, and the resulting set operations frequently dominate end-to-end runtime. In existing sparse bit-vector representations, we find that the main cost is not bitmap arithmetic but irregular index matching: identifying allocation-site representatives shared by two operands before combining their corresponding blocks. We accelerate this kernel by recasting index matching as a branch-free SIMD primitive. The key is a sparse layout that stores allocation-site indices and their blocks in contiguous arrays, enabling parallel scans over indices and masked SIMD updates of blocks while preserving sparsity. We evaluate the technique on 12 real-world programs under both context-insensitive and context-sensitive variants. Compared to a conventional sparse bit-vector baseline, our approach reduces analysis time by up to 3.8× (geomean: 2.0×) and peak memory by up to 86% (geomean: 58%), with larger gains for more precise analyses. The improvements persist across alternative set representations and constraint solving strategies.

Article Search Article: ase26main-p176-p doi:10.1145/3832783.3837421

DesignFix: Context-Augmented Repair of Design Rule Violations
Mainul Hossain and Thomas D. LaToza
(George Mason University, USA)
Design rules make project-specific design decisions explicit and checkable, helping teams preserve architectural intent as systems evolve. Although automated checkers can detect violations, developers must still interpret the rule, locate the code that must change, and construct a repair consistent with project conventions. We present DesignFix, an IDE-integrated approach that turns checker-reported violations into project-grounded, reviewable repairs. DesignFix augments each violation with the rule description, a rule-satisfying example from the same project, and focused structural context, while a two-step LLM pipeline separates repair reasoning from structured edit extraction.
We report three complementary evaluations. A systematic study of seven prompt configurations across 9,450 repair trials found that the full workflow achieved up to 90% repair success, 89% explanation correctness, and 90% preservation of unrelated code: 26 percentage points above a baseline prompt. We conducted a controlled experiment with 14 developers. Compared to a control condition with browser-based LLMs, developers completed repairs 36% faster with DesignFix, increased correct repairs from 43% to 93%, and reported significantly lower workload on five of six NASA-TLX dimensions. Finally, we ran the autonomous in-IDE agent Cline on six repair task types drawn from our user study. Cline produced a correct repair for only half of them, while averaging 12.1× DesignFix’s token use and substantially higher API cost per run. Together, these results suggest that effective design-rule repair depends on grounding the LLM in checker-derived, project-specific design intent and on keeping repairs under developer review, needs that repository autonomy alone does not meet.

Article Search Article: ase26main-p184-p doi:10.1145/3832783.3837422

LaMAR: Latent Multi-agent Collaboration via KV-Cache Communication for Automated Program Repair
Rui Jiao, Yue Zhang, Zhexuan Feng, and Jinku Li
(Xidian University, China; Shandong University, China)
Multi-agent systems for automated program repair communicate through text tokens, requiring each agent to decode its internal state into discrete symbols and the next agent to re-encode them. This conversion loses information and incurs token overhead that grows quadratically with the agent count. We develop Latent Agent Communication (LAC), a theoretically grounded framework that formalizes how agents can share KV-cache representations directly in continuous space, and prove it provides a communication channel with strictly higher information capacity while reducing token complexity from O(N^2) to O(N). We instantiate LAC as LaMAR, a dual-chain repair system: Bug Analysis Chain diagnoses the bug through four collaborating latent agents; Repair Chain refines the patch through multi-agent latent collaboration with cross-chain context transfer, and supports optional multi-round iteration for complex defects. Evaluated across three benchmarks spanning Java, Python, and C++ with three model families, LaMAR with Qwen3-8B achieves 24.5% repair success rate on Defects4J and 62.1%--87.5% on DebugBench and QuixBugs, outperforming all baselines in the large majority of settings. Ablation experiments identify the bug analysis chain and latent communication as the most critical components.

Article Search Article: ase26main-p198-p doi:10.1145/3832783.3837423

Enhancing Trace-Based Root Cause Analysis for Microservice Systems via Code Change Understanding
Min Zhang, Chenxi Zhang, Senyu Xie, Shihong Chen, Lei Wu, and Xin Peng
(Fudan University, China; Xidian University, China; McDonald's, China)
Microservice architecture has become the dominant design pattern for modern web applications. However, the dynamic structure, complex inter-service invocations, and continuous updates of microservices make root cause analysis (RCA) particularly challenging. Existing RCA methods typically rely on runtime monitoring data such as traces and metrics, and adopt techniques including deep learning, graph analysis, and spectrum analysis. However, most of these methods overlook software changes, even though code and deployment updates are often the underlying causes of production failures. This paper introduces the idea of RCA left-shifting, which shifts part of the diagnostic effort to the development stage by proactively understanding change semantics before failures occur. Based on this insight, we propose DeltaRCA, a trace-based RCA approach that integrates large language model (LLM)-driven change impact analysis with spectrum-based root cause localization. DeltaRCA uses an LLM agent to identify semantically affected operations from code changes. This information is incorporated into a heterogeneous graph to guide the weight propagation process in spectrum-based fault localization. We evaluate DeltaRCA on two microservice benchmarks with 200 injected fault cases. The results show that DeltaRCA outperforms state-of-the-art methods, achieving 63% Top-1 and 79% Top-3 accuracy on average. Furthermore, ablation analysis demonstrates that incorporating change understanding significantly improves localization precision.

Article Search Article: ase26main-p199-p doi:10.1145/3832783.3837424

Assessing Uncertainty in Performance Modeling: Conformal Prediction vs. Bayesian Regression
Stefan Jahns, Johannes Dorn, Max Weber, Sven Apel, and Norbert Siegmund
(Leipzig University, Germany; Saarland University, Germany)
Most software systems today are configurable. As a consequence, predicting performance is inherently challenging due to the sheer number of configurations exhibiting individual performance variations. When predicting the performance of unseen configurations, it is imperative to know how reliable such predictions are. Bayesian methods have been shown to be able to quantify the uncertainty in their estimates, but require prior information about performance distributions and, more importantly, are constrained in their learning technique, effectively limiting their practical applicability. Conformal prediction, an entirely novel approach to uncertainty quantification in this area, augments existing learning algorithms via a statistical post-hoc analysis. This allows us to provide uncertainty estimates even for point-estimate algorithms, such as random forests or deep neural networks. The goal of this study is to evaluate how assumptions of conformal predictions regarding sample size and distribution, as well as the heterogeneity of learning algorithms, affect their applicability to software performance prediction. We conduct a comprehensive study comparing four conformal prediction variants against P4, the current standard for uncertainty-aware performance prediction, analyzing accuracy, scalability, and uncertainty quantification. Our results show that conformal prediction consistently outperforms the state of the art across six out of eight real world scenarios, producing correct confidence intervals that are 46–75 % narrower, and thus more certain. Notably, conformal prediction can be used to enrich existing techniques, with post-hoc uncertainty quantification in a plug and play fashion.

Article Search Article: ase26main-p207-p doi:10.1145/3832783.3837425

RACE-Bench: A Reasoning-Augmented Benchmark for Repository-Level Code Agents on Feature Addition
Shuhan Liu, Zhiyi Zhao, Xing Hu, Kui Liu, Xiaohu Yang, and Xin Xia
(Zhejiang University, China)
Repository-level code agents have shown strong promise in real-world feature addition tasks, making reliable evaluation of their capabilities increasingly important. However, existing benchmarks primarily evaluate these agents as black boxes based on final test correctness, providing limited insight into how they reason and where failures arise. To address this limitation, we introduce RACE-Bench, a reasoning-augmented benchmark for evaluating code agents on repository-level feature addition tasks. RACE-Bench contains 528 real-world feature addition instances from 12 open-source repositories. Each instance is paired with executable patch verification and structured intermediate reference reasoning covering issue understanding, file localization, implementation tasks, and step decomposition. Based on this design, we introduce a dual-track evaluation framework that jointly measures patch correctness and intermediate reasoning alignment with developer-accepted reference trajectories. We evaluate three representative repository-level code agents on RACE-Bench. On the full benchmark, Resolved Rate ranges from 29% to 70% across different agents. Our reasoning-level analysis further shows that while current agents perform well at understanding high-level intent, their performance degrades substantially when translating intent into concrete implementation steps. We also find patches that can be applied but still fail the tests cover fewer reference-reasoning elements (35.7% lower recall) and contain more unsupported reasoning elements (94.1% higher over-prediction) than successful patches. These findings highlight the importance of evaluating repository-level code agents beyond final patch correctness by examining the quality of their reasoning processes.

Article Search Article: ase26main-p208-p doi:10.1145/3832783.3837426

How Effective Are NPM Malicious Package Detectors? A Large-Scale Empirical Study
Wenbo Guo, Zhongwen Chen, Zhengzi Xu, Chengwei Liu, Ming Kang, Shiwen Song, Chengyue Liu, Yijia Xu, Weisong Sun, and Yang Liu
(Nanyang Technological University, Singapore; Sichuan University, China; Imperial Global Singapore, Singapore; Nankai University, China; Singapore Management University, Singapore)
The NPM ecosystem faces escalating threats from malicious packages that exploit its open publication model. While numerous detection tools have been proposed, they are evaluated on disparate datasets with inconsistent settings, making cross-tool comparison unreliable and leaving practitioners without clear guidance.
We present the first large-scale empirical study of NPM malicious package detection, evaluating 11 tools with 16 variants on a unified benchmark of 6,420 malicious and 7,288 benign packages annotated with 11 behavior categories and 8 evasion techniques. Unlike prior work, we inspect each tool’s source code to explain why tools succeed or fail, not merely how often. Our key findings: (1) the precision and recall a tool achieves are structurally determined by how it resolves the ambiguity between code capability and malicious intent, with IntelGuard reaching the best F1 at 95.98% by grounding its judgment in retrieved evidence and GuardDog the best among conventional tools at 93.32%; (2) behavioral coupling amplifies detection signals when behaviors co-occur, raising SAP_DT from 3.2% to 79.3% for the collect-and-exfiltrate chain; (3) 80.3% of malware uses no evasion because the ecosystem lacks mandatory pre-publication scanning; (4) ML degradation is driven by concept convergence rather than concept drift, since malware became simpler and every decision boundary fitted to a corpus ages with it; (5) combination effectiveness equals complementarity minus false-positive introduction, not paradigm diversity. Strategic combinations reach up to 97.21% accuracy and 97.02% F1. We release our benchmark and evaluation framework.

Article Search Article: ase26main-p218-p doi:10.1145/3832783.3837427

Localizing Data Annotation Faults in Semantic Segmentation Systems
Pengyi Yang, Teng Long, Zhiwu Xu, and Linhai Jing
(China University of Geosciences (Beijing), China; Shenzhen University, China)
Data annotation fault localization is a critical software quality assurance task in deep learning pipelines, as annotation faults can severely compromise model reliability and system dependability in safety-critical applications. Semantic segmentation requires assigning a class label to every pixel in the image, making it one of the most fault-prone annotation tasks, with faults ranging from incorrect class labels to imprecise boundaries. However, existing annotation fault localization methods target simpler annotation types such as classification and object detection, and rely on models trained on potentially noisy data, making them inadequate for dense annotations such as semantic segmentation. We adapt spectrum-based fault localization from program elements to annotation components in semantic segmentation, and propose SegFaultLoc, an annotation fault localization framework that identifies and ranks faulty annotation components by suspiciousness without segmentation model training. SegFaultLoc decomposes annotation masks into connected components and extracts noise-independent features from frozen pretrained encoders to construct dual fault spectra: a semantic spectrum for classification and missing faults, and a geometric spectrum for localization and fragmentation faults. Learning-based suspiciousness aggregation integrates these signals into unified suspiciousness scores for prioritized inspection. Systematic experiments on three benchmarks against seven data-quality baselines demonstrate that SegFaultLoc achieves up to 9.2% improvement in APFD, 21.9% in Precision@10%, and 22.4% in nDCG@10% over the best baseline. Validation on naturally occurring annotation faults in the real-world COCO-Stuff dataset further confirms that correcting detected faults consistently improves segmentation performance.

Article Search Article: ase26main-p227-p doi:10.1145/3832783.3837428

“Impossible to Hide Secret ...”: Uncovering Security and Privacy Issues in LLM-Native IDEs
Mostafijur Rahman Akhond, Md Afif Al Mamun, Gias Uddin, and Song Wang
(York University, Canada; University of Calgary, Canada)
LLM-native IDEs (Integrated Development Environments), aka LIDEs, are designed from the ground up to work with Large Language Models (LLMs). LIDEs have found remarkable success in Software Engineering (SE) tasks such as coding, debugging, and program comprehension. LIDEs are software systems, and, like any system, they can exhibit vulnerabilities. In this paper, we study the security and privacy issues that developers reported while using popular LIDEs in their development tasks. We collected 1.1M posts from 29 popular subreddits related to LIDEs. We identified 446 posts and analyzed over 6K comments to the posts that discussed security and privacy issues in almost all popular LIDEs, such as Cursor, Copilot, Codex, etc. Using a mix of qualitative and quantitative methods, we constructed a taxonomy of the reported security and privacy issues. Our results show that most issues in LIDEs stem from system-level design choices, rather than the underlying LLMs, such as user data access, unchecked autonomous actions, etc. To overcome these issues, developers frequently relied on external safeguards like code sandboxing and manual reviewing, highlighting prevalent mistrust among developers about LIDEs. We share lessons from our study to support future design of secure and privacy-aware LIDEs.

Article Search Article: ase26main-p234-p doi:10.1145/3832783.3837429

Smart Contract Synthesis via Multi-modal Specifications
Tanglin Chen, Haoxian Chen, Huilin Xiang, and Yuepeng Wang
(ShanghaiTech University, China; Simon Fraser University, Canada)
Smart contracts manage substantial digital assets on blockchains. Given their transparency and immutability, ensuring their correctness before deployment is essential. While verification tools offer assistance, the iterative nature of the process presents hurdles, especially for less experienced developers. In this paper, we explore a paradigm shift in smart contract development: writing high-level specifications instead of low-level code. To realize this goal, we propose SmartSpec, a synthesizer that generates smart-contract implementations from functional and safety specifications and validates them with integrated verification. To avoid overly restrictive outcomes, SmartSpec employs maximally permissive synthesis, ensuring contracts preserve all safety requirements while remaining as permissive as possible. Its novel algorithm combines deductive and inductive synthesis techniques, balancing precision and generalization. On 27 contracts, synthesis runs in seconds to minutes; 23 match references without interactive refinement (85%), while the remaining four can be corrected with up to four rounds of interactive feedback through extra examples. Mean gas overhead vs. references on the table is +0.7%.

Article Search Artifacts Available Article: ase26main-p244-p doi:10.1145/3832783.3837430

BDiff: Block-Aware and Accurate Text-Based Code Differencing
Yao Lu, Wanwei Liu, Tanghaoran Zhang, Kang Yang, Yang Zhang, Wenyu Xu, Longfei Sun, Xinjun Mao, Shuzheng Gao, and Michael R. Lyu
(National University of Defense Technology, China; Chinese University of Hong Kong, China)
Code differencing is a fundamental technique in software engineering. Traditional line-level differencing tools (e.g., Git Myers algorithm) cannot accurately identify block-level edit actions (EAs) such as block move and copy, which are common in real development. They split multi-line block edits into fragmented line-wise changes, raising developers’ cognitive overhead during code review. This paper presents BDiff, a text-based differencing algorithm capable of identifying two types of block-level EAs and five types of line-level EAs. Building on traditional differencing algorithms, we first construct a candidate set containing all possible line and block mappings. Leveraging the Kuhn-Munkres algorithm, we then compute the optimal mapping set and derive a minimal edit script (ES) that closely aligns with the original developer intent. To evaluate BDiff, we selected five state-of-the-art tools, including large language models (LLMs), as baselines, and adopted a combined qualitative and quantitative approach to assess their performance in terms of ES size, result quality, and running time. Experimental results show that BDiff produces higher-quality differencing results than baseline tools while maintaining competitive runtime performance. Our experiments also show the unreliability of LLMs in code differencing tasks and their infeasibility in terms of runtime efficiency. Based on the proposed algorithm, we have implemented an open-sourced visual differencing tool.

Article Search Artifacts Available Article: ase26main-p251-p doi:10.1145/3832783.3837431

PCodeTrans: Translate Decompiled Pseudocode to Trace-Level Equivalent Source Code
Yuxin Cui, Zeyu Gao, Siliang Qin, Yuanda Wang, Jiaming Wang, Xize Luo, Peng Zhou, and Chao Zhang
(Tsinghua University, China; Institute of Information Engineering at Chinese Academy of Sciences, China; Peking University, China; Hunan University, China)
Decompilation remains foundational for binary analysis, yet conventional tools prioritize human readability over compilation success and test-validated runtime consistency. While recent large language model approaches attempt refining decompiled pseudocode, they often introduce semantic hallucinations. Operating statically without dynamic feedback or attempting whole binary repairs lacking scalability, these existing methods struggle to resolve genuine runtime divergences. We present PCodeTrans, a dynamic feedback driven framework that implements a loop of recompilation, validation, and alignment for the recovery of functional source code. Extracting a minimal yet globally consistent context, PCodeTrans employs an in situ substitutable engine. This mechanism injects isolated compiled functions directly into the original binary, natively preserving authentic execution environments. Utilizing trace-level alignment as a diagnostic objective, PCodeTrans leverages breakpoint matched differential tracing, generating precise runtime causal diagnostics that guide the model through iterative refinement resolving behavioral discrepancies. Evaluated across a comprehensive Linux and Windows Benchmark suite including Coreutils, Binutils, PuTTY, and SQLite3, PCodeTrans achieves substantial translation performance refining raw Hex-Rays outputs. Empirical results confirm PCodeTrans achieves 100% function-level compilation success and 99.5% test-validated consistency on unstripped binaries, resolving up to 85.6% of logic errors. Even for stripped executables, the framework maintains over 97% test-validated consistency, narrowing the gap between raw decompilation and functional source code. Surpassing existing baselines, PCodeTrans paves a practical path toward recovered source that can be recompiled, validated to pass test harnesses, and consumed by source-level security analysis and fuzzing tools.

Article Search Article: ase26main-p256-p doi:10.1145/3832783.3837432

AlphaCu: A Transformation-Driven Synthesis Framework for LLM-Based GPU Kernel Generation
Yangjie Zhou, Xing Ma, Zihan Liu, Qing Wang, Weihao Cui, Wu Sun, Hanjing Wang, Binhang Qi, Jingwen Leng, Yun Lin, Minyi Guo, and Jin Song Dong
(National University of Singapore, Singapore; Shanghai Jiao Tong University, China; Shanghai Qi Zhi Institute, China; Nanjing University, China; Harvard University, USA)
Large language models show promise for code generation, yet their effectiveness in producing high-performance GPU kernels remains insufficiently understood. Recent agentic workflows improve over direct prompting through specialized generation roles and execution feedback, but our characterization of a representative workflow reveals failures in optimization-strategy selection, hardware-specific instruction realization, and shape-dependent parameter selection.
We present AlphaCu, a transformation-driven synthesis framework for LLM-based GPU kernel generation. AlphaCu treats kernel generation as structured synthesis over explicit optimization decisions rather than end-to-end code emission. Specifically, AlphaCu represents these decisions as scoped transformation passes maintained across agent-coordinated synthesis stages, enabling compatibility checking and failure attribution. AlphaCu binds selected hardware-sensitive operations to pre-validated microkernels and uses target-device measurements to revise transformation choices and tune runtime configurations.
We evaluate AlphaCu on 104 test cases covering single and fused operators across four GPU architectures: NVIDIA H100, A100, V100, and AMD MI210. Under comparable generation budgets, AlphaCu achieves 100% functional correctness on our benchmark suite and delivers an average speedup of 5.64× over prior LLM workflows, while remaining competitive with vendor libraries and tensor compilers.

Article Search Article: ase26main-p287-p doi:10.1145/3832783.3837433

How Do LLMs Read Bug Reports? An Empirical Study of Attention in LLMs for Automated Program Repair
Ramtin Ehsani, Irene Manotas, Saurabh Pujar, Luca Buratti, and Preetha Chatterjee
(Drexel University, USA; IBM Research, USA)
Large Language Model (LLM)-based Automated Program Repair systems are advancing rapidly, yet their performance remains inconsistent. Even when provided with the same contextual information, an LLM may generate a correct patch for one bug but fail on another closely related bug. Why this happens remains poorly understood, and it is unclear how LLMs prioritize the diverse information in bug reports and whether model attention affects repair success. In this paper, we present the first empirical study of attention patterns in LLM-based program repair, providing interpretable insights into how models process bug reports and where their attention is concentrated during repair. We analyze 319 real-world Python and Java bugs from SWE-bench Verified and Multi-SWE-bench to study (RQ1) how model attention is distributed across bug report sections, (RQ2) how attention patterns within each section differ between successful and unsuccessful repairs, and (RQ3) how these patterns compare to information developers consider important for bug fixing. We find that successful repairs are characterized by diffused attention across multiple diagnostic components such as bug descriptions, stacktraces, and test cases, while failures often exhibit over-localized attention toward metadata such as version information. We further observe that stronger alignment between model attention and developer-identified key sections and phrases is associated with higher repair success. Our results provide the first empirical evidence that attention misallocation is a key factor in LLM-based APR failures, and offer actionable insights for designing more interpretable and reliable future APR systems.

Article Search Artifacts Available Article: ase26main-p306-p doi:10.1145/3832783.3837434

Mind the Gap: Do Widely-Used SAST Tools Really Cover the Language-Specific Top CWEs?
Zhilin Li, Yiran Nie, Xianglong Qi, Jiwei Yan, and Jun Yan
(Institute of Software at Chinese Academy of Sciences, China; Hangzhou Institute for Advanced Study at University of Chinese Academy of Sciences, Hangzhou, China; Beijing University of Technology, China)
Static Application Security Testing (SAST) tools play a key role in identifying vulnerabilities early in the development cycle. While these tools claim to detect a wide range of CWEs, their actual coverage is often unclear. In particular, due to language-specific characteristics, it is unclear to what extent SAST tools cover the CWEs most relevant to a given programming language, creating a gap between tool capabilities and developers’ security needs. To address these questions, this experience paper presents a large-scale empirical investigation into the gap between SAST Tools and Language-Specific Top CWEs. Using a heuristic labeling approach on 197,016 CVEs reported over the past decade, we construct the Top_Lan CWE Lists for three widely used programming languages: C/C++, Java, and Python. Based on these datasets, we evaluate nine popular SAST tools to identify coverage gaps and tool-specific performance characteristics.
Our study yields several key findings. First, the language-specific Top_Lan CWE lists differ a lot from the globally ranked CWE list, indicating that SAST tools need to focus on different types of vulnerabilities for different languages. Second, the whole SAST tool set now exhibits low coverage of the Top_Lan CWE list, highlighting the need for more targeted code checkers. Third, the covered Top_Lan CWE sets for different SAST tools vary substantially, underscoring the need to select and combine tools based on vulnerability types. Finally, for top CWEs shared across multiple languages, existing multi-language SAST tools may support detection in only one language, suggesting opportunities to enhance coverage by migrating existing checkers. These findings reveal fundamental challenges for static analysis in detecting certain types of vulnerabilities and provide insights for improving future SAST tool design. They offer practical guidance for practitioners in selecting and combining tools to detect language-specific vulnerabilities more effectively.

Article Search Artifacts Available Article: ase26main-p316-p doi:10.1145/3832783.3837435

Smart Brain: Semantic Anomaly Detection for Operational Time Series in Large Scale Service Systems
Hang Cui, Zexin Wang, Jingjing Li, Juncheng Hu, Haotian Si, Cenjie Hu, Quan Zhou, Yongchang Hu, Lei Han, Dan Pei, Changhua Pei, and Gaogang Xie
(University of Chinese Academy of Sciences, China; Computer Network Information Center at Chinese Academy of Sciences, China; Jilin University, China; Independent Researcher, China; Shenyang Institute of Automation at Chinese Academy of Sciences, China; Huawei Technologies, China; Tsinghua University, China)
Time series anomaly detection is a core component of operational automation in large scale service systems. However, most existing methods model anomalies as numerical deviations in the data space, while real software operations require decisions over whether an observed behavior constitutes an actionable abnormal event. This mismatch often leads to brittle thresholds, excessive false alerts, and outputs that are difficult for operators to interpret and act upon. In this paper, we argue that practical anomaly detection should move from point-centric data anomaly detection to semantic anomaly detection. We present Smart Brain, a two stage framework for semantic anomaly detection in operational time series. Stage I extracts structured temporal evidence by recognizing interpretable anomaly patterns. Stage II combines the extracted evidence with metric semantics and operator authored rules to produce semantic anomaly decisions and human-readable explanations. Through evaluation on benchmark datasets and real industrial monitoring scenarios, we show that Smart Brain improves operationally meaningful anomaly identification while providing stronger interpretability and deployment suitability than score centric pipelines. Beyond the framework itself, our study offers practical lessons for building anomaly detection systems in production software environments, highlighting the importance of semantic grounding, structured evidence, and lightweight adaptation to evolving services.

Article Search Article: ase26main-p337-p doi:10.1145/3832783.3837436

AgentChaos: Chaos Engineering for Agent Systems via Programmatic Fault Injection
Gou Tan, Zhensu Sun, Jieke Shi, Ting Zhang, Zilong He, Qingfu Wu, Shuai Liang, Weifeng Sun, Junda He, Pengfei Chen, Chuanfu Zhang, Lwin Khin Shar, and David Lo
(Sun Yat-sen University, Guangzhou, China; Singapore Management University, Singapore; Monash University, Australia; China Unicom Software Research Institute, Beijing, China)
Agent systems rely on LLM APIs for every response, but these APIs can return server errors, truncated responses, or corrupted content that propagates through downstream agents and causes task failure. Evaluating robustness under these faults is crucial for reliable deployment. Existing fault injection methods are offline, require source code modification, or cannot modify specific response fields. A comprehensive evaluation also requires a systematic fault taxonomy because different fault types affect downstream agents differently. We propose AgentChaos, a chaos engineering framework for controlled, runtime, non-intrusive LLM API fault injection. Since all agent systems access LLMs through the same HTTP interface, we inject faults at this shared layer without modifying source code. We define crash, omission, and value faults on content and tool call fields, intercept and modify LLM API responses at runtime, and verify whether each fault is triggered to filter untriggered tasks and avoid underestimating fault impact. Evaluations across agent systems, benchmarks, and backbone LLMs under 65 fault configurations show that all systems degrade under fault injection, with pass@1 dropping by up to 50 percentage points. The ranking is consistent across models, suggesting that robustness depends on system implementation rather than model capability. Existing fault diagnosis methods achieve below 53% accuracy on fault type and below 56% on fault step, leaving room for improvement. We further reveal practical findings for agent system developers.

Article Search Artifacts Available Article: ase26main-p355-p doi:10.1145/3832783.3837437

Precise and Efficient Static Data Race Detection
Pei Wang, Zhihang Sun, and Fei He
(Tsinghua University, China)
Data races pose a serious threat to the correctness of concurrent programs. Traditional model-checking-based race detection techniques typically encode race detection as assertion verification, but this introduces substantial overhead due to numerous auxiliary variables and statements. In this paper, we present a precise modeling of data races based on event-level ordering constraints, which avoids any instrumentation while capturing the exact semantics and covering all potential race occurrences within the given bounded scope. We integrate it into a concurrent program verification framework and prove its correctness. Moreover, we enhance the existing theory solvers for ordering consistency theory with two new components, enabling effective reasoning about data races. Finally, we introduce an encoding optimization that leverages the novel concept of value-critical events to eliminate redundant value constraints. We implement our approach in a prototype tool, Deagle-DR, and compare it against state-of-the-art tools, including Dartagnan, Ugemcutter, Cseq-DR, LaDR, and Iekke. Experiments show that our approach is competitive with state-of-the-art methods.

Article Search Artifacts Available Article: ase26main-p369-p doi:10.1145/3832783.3837438

Best-Effort GR(1) Synthesis: Exploiting Environmental Cooperation under Unrealizability
Sirui Liu, Yating Zhang, Chi Hu, and Wei Dong
(National University of Defense Technology, China; China Academy of Engineering Physics, China)
Reactive synthesis automatically derives correct-by-construction controllers from temporal logic specifications. GR(1) synthesis is widely adopted due to its polynomial-time complexity and practical expressiveness. In practice, however, a significant proportion of GR(1) specifications are unrealizable. Conventional synthesis tools report failure in such cases, yet real-world environments are often governed by their own objectives rather than purely adversarial behavior, presenting opportunities for the system to leverage environmental cooperation toward satisfying its justice guarantees.
For unrealizable GR(1) specifications, this paper proposes best-effort GR(1) synthesis. The approach requires the system strategy to satisfy two properties: best-effort extensibility, which preserves at every state reached by the play the possibility of satisfying all system justice guarantees, and persistent satisfiability of environment assumptions, which ensures that environmental cooperation does not preclude the environment from satisfying its own justice assumptions. For unrealizable but well-separated specifications, we construct a 3-nested µ-calculus fixpoint formula that characterizes the best-effort system winning region and prove its soundness and completeness. The resulting algorithm performs O(n|Q|) symbolic next-state computations, compared with O(mn|Q|²) for standard GR(1) synthesis. We further analyze the relationship between the best-effort and standard GR(1) winning regions through finite cooperative reachability, yielding a hierarchical classification of dependence on environmental cooperation at the region and initial-state levels for well-separated but unrealizable specifications, thereby quantifying the degree of unrealizability.
We implement best-effort checking, controller construction, and classification in the Spectra GR(1) synthesizer and evaluate the approach on a diversified corpus of SYNTECH, AMBA, and GenBuf specifications. Experimental results demonstrate that best-effort realizability is common among well-separated but unrealizable specifications and that all top-level unrealizability categories occur in the combined benchmark corpus.

Article Search Artifacts Available Article: ase26main-p372-p doi:10.1145/3832783.3837439

TELLER: Non-intrusive Cross-Layer Root-Cause Analysis for LLM Inference
Ruilin Xu, Junyi Li, Pengfei Chen, and Zongxuan Xie
(Sun Yat-sen University, China)
Large language model (LLM) inference has evolved from an offline workload into a continuously operated software service, yet root-cause analysis remains difficult because a single request spans the inference engine, Python/C++ backend, host CUDA APIs, GPU kernels, and distributed communication. Existing profilers expose raw timelines, while log-based diagnosis often misses cross-layer execution semantics and request-level structure. We present TELLER, a non-intrusive trace- and log-aware LLM inference root-cause analysis framework. TELLER first collects NVTX/CUPTI traces and service logs without modifying model binaries, then reconstructs per-request call-chain trees and aligns log lines with the corresponding execution steps. We introduce a dependency-aware causal-context slice that preserves parent-child structure, temporal order, and communication relations, and a Trace Pair Encoding (TPE) tokenizer that compresses such slices into compact structural token sequences with parent, depth, and duration attributes. On top of these representations, TELLER combines numeric candidate localization with a multimodal root-cause model that jointly predicts abnormal steps, localizes suspicious operators, and generates natural-language explanations. Experiments on multi-node GPU inference workloads show a clear compression-accuracy trade-off: a moderate TPE vocabulary reduces per-step trace length by more than 80% while achieving the best overall performance on both horizontal (cross-node communication) and vertical (within-node execution stack) views, whereas more aggressive compression substantially degrades diagnosis quality. Further analyses under low-fault priors, strengthened baselines, modality ablations, explanation-quality checks, and tracing overhead show that TELLER provides a practical triage and evidence-localization substrate for LLM inference RCA.

Article Search Artifacts Available Article: ase26main-p423-p doi:10.1145/3832783.3837440

Predicate-Guided Synthesis of Input Generators via Gentoo
Vasudev Vikram and Rohan Padhye
(Carnegie Mellon University, USA)
Modern generator-based fuzzing techniques combine lightweight input generators with coverage-guided mutation as a method of exploring deep execution paths in a target program. A complementary approach in prior research focuses on creating highly customized, domain-specific generators that encode structural and semantic logic sufficient enough to reach deep program states; the challenge comes from the overhead of writing and testing these complex generators. We investigate whether AI coding agents can automatically synthesize such target-specific generators, and whether the resulting generators are strong enough to obviate the need for coverage guidance and mutation entirely. Our approach, Gentoo, is comprised of an LLM coding agent (provided terminal access and source code of the fuzz target and its library) instructed to iteratively synthesize and refine an input generator, and optionally provided fine-grained predicate-level coverage feedback. We evaluate three configurations of Gentoo against human-written generators on fuzz targets for 7 real-world Java libraries. Our findings show that agent-synthesized generators achieve statistically significantly higher branch coverage than human-written baseline generators on 4 of 7 benchmarks. Critically, the use of coverage guidance and mutation strategies is not statistically significantly beneficial for agent-synthesized generators, but is significant for all human-written generators, suggesting that structural and semantic logic encoded in the agent generators makes coverage guidance largely unnecessary.

Article Search Article: ase26main-p430-p doi:10.1145/3832783.3837441

DepUpgrade: Automating Dependency Upgrade through State-Path Exploration
Xiangxi Ma, Yifan An, Wentong Tian, Xuanqi Wang, Qingao Dong, Xiang Gao, and Hailong Sun
(Beihang University, China)
Third-party dependencies frequently introduce API breaking changes, and upgrading them causes compilation or runtime failures in client applications. Existing approaches have explored automating dependency upgrades, but struggle with fixing complex issues. This is mainly because traditional methods lack semantic adaptability, while recent prompt-only LLM-based migration approaches often suffer from hallucinations and fail to navigate multi-step, cascading repairs. To solve this problem, in this paper, we propose DepUpgrade, a framework that models dependency upgrade as a state-path exploration problem. A state is a snapshot of the client codebase at a particular point during migration. Transitions between states are validated via compiler feedback and rolled back if regressions occur. Instead of performing a single-pass generation, DepUpgrade systematically explores a search space of intermediate states with the goal of reaching a successful state. DepUpgrade achieves this goal in three steps: API migration knowledge extraction, repair task formulation and state-path exploration. We evaluate DepUpgrade on a benchmark of 123 migration tasks and 18 real-world projects. Experimental results demonstrate that our framework achieves an unprecedented 96.7% full repair rate on the benchmark, outperforming other baselines. On the real-world projects, DepUpgrade fully repairs 15 of the 18 projects and leaves 31 of the 1,085 initial compilation errors unresolved.

Article Search Article: ase26main-p442-p doi:10.1145/3832783.3837442

PWCC: Probability-Weighted Combinatorial Coverage for Risk-Informed Scenario Testing of Autonomous Systems
Borhaneddine Hamadou, Chokri Mraidha, and Nikolaos Matragkas
(Université Paris-Saclay, France; CEA List, France)
Scenario-based testing of autonomous systems faces a fundamental unsolved problem: knowing when to stop. Existing coverage metrics measure either structural diversity or combinatorial breadth, but neither tells you how much of the system’s expected real-world operation remains untested. A test suite can score well on every existing metric while systematically missing the conditions the system will actually encounter.
We introduce Probability-Weighted Combinatorial Coverage (PWCC), a metric that simultaneously measures coverage of the Op- erational Design Domain (ODD) and test adequacy. PWCC learns a probabilistic model of expected operating conditions from oper- ational profiling data, uses it to assign probability mass to 𝑡-way combinations of scenario features, and declares testing adequate when the uncovered mass falls below a risk-calibrated threshold. This gives practitioners something no existing metric provides: a principled, quantitative answer to “have we tested enough?”
Evaluated on four autonomous driving datasets against seven baselines, PWCC demonstrates three properties. First, it is the strongest predictor of fault detection where the operational dis- tribution carries signal, reaching Spearman 𝑟𝑠 = +0.981 on Deep- Collision — 0.167 above the best combinatorial baseline at a lower interaction order. Second, it suppresses false sufficiency: on Deep- Scenario, PWCC achieves FSR = UFC = 0 across all splits versus CC’s mean UFC ≥ 95.8%, false sufficiency eliminated entirely, and, judged at its own stopping point, surfaces 99.6% of failure-triggering com- binations where combinatorial saturation stops after 6.4% of the campaign; on failure-rich datasets UFC drops by up to 4.9×. Third, it is stable: the score varies by at most 0.006 across a ten-fold increase in Monte Carlo sample size and grows monotonically with suite size.
These results show that operational probability weighting is the decisive factor in avoiding dangerous false sufficiency verdicts; and that PWCC transforms coverage from a descriptive statistic into a risk-informed stopping decision.

Article Search Artifacts Available Article: ase26main-p443-p doi:10.1145/3832783.3837443

Lost in Translation, Found in Summary: Summary-Driven Supervision for Cross-Language Code Retrieval
Zhijie Liu, Yisi Liu, Linxi Jiang, Yutian Tang, Lili Xiao, and Zhipeng Xue
(Independent Researcher, China; Donghua University, China; University of Glasgow, UK; Zhejiang University, China)
Cross-language code retrieval (CLCR) is critical to modern software development, and recent studies have shown that contrastive learning is an effective way to learn cross-language code semantics. However, these methods are still limited by two fundamental challenges. First, they rely on online judge platforms or multilingual NL-to-code datasets, which either lack diversity or fail to ensure semantic alignment between code snippets, thereby limiting the effectiveness of cross-language semantic learning. Second, during training, code snippets from different languages are often encoded into heterogeneous representation distributions. This distribution inconsistency introduces similarity bias and weakens the model's ability to capture accurate cross-language semantics. To address these challenges, we propose SemSIR, an end-to-end framework for CLCR. The key insight is to use Summary IR, a natural-language intermediate representation, to bridge semantic gaps across languages. Built on this insight, SemSIR automatically constructs scalable cross-language supervision from real-world repositories, and further combines distribution alignment with CLIP-style contrastive training to learn more discriminative language-independent code semantics. In our evaluation, we collect 24 million real-world code snippets across 9 languages and automatically construct 7 million semantic groups. Manual inspection of 1,000 sampled groups shows that 92.7% are fully semantically coherent, with an average Group Purity of 96.8%. On CLCR, SemSIR consistently outperforms all baselines on two benchmarks, surpassing the state-of-the-art method by up to 24.7% in MRR and 50.1% in MAP on average. In addition, experimental results on a security case study of tracing rewritten malware across languages show that SemSIR achieves 92.20 MRR and 88.40 MAP, highlighting its potential for security-oriented CLCR.

Article Search Article: ase26main-p479-p doi:10.1145/3832783.3837444

ExeCRE: Execution-Consistency Guided Reliability Estimation for Self-Correcting Code Generation
Yiru Dong, Richong Zhang, Fanshuang Kong, and Si Chen
(Beihang University, China)
Large language models (LLMs) have made notable progress in code generation, but they still struggle on challenging tasks that require sophisticated algorithms or complex implementations. Recent methods increasingly use code execution as feedback, especially in self-correction pipelines that construct verification signals from generated code. However, these pipelines often depend on supervision signals whose reliability is unknown, which can introduce misleading feedback, unnecessary revisions, and incorrect final answers. To address this issue, we propose ExeCRE, an Execution-Consistency guided code Reliability Estimation framework. Instead of judging candidate code by tests or LLM feedback, ExeCRE estimates code reliability by statistically analyzing consistency patterns in execution outputs over a large number of randomly generated inputs. It collects execution outputs over generated inputs, projects them into consistency signals, and applies Dawid–Skene model to infer latent code reliability. We integrate ExeCRE into self-correction for code generation. Experiments show that ExeCRE consistently improves both effectiveness and stability, while substantially reducing misleading correction signals. Under GPT-5.2 on LiveCodeBench, the average number of misleading feedback cases on already correct code drops from 113.2 with a representative self-correction baseline to 14.0 with ExeCRE. As an additional study, we apply the same reliability estimation strategy to code-based mathematical reasoning and observe similar benefits. These results suggest that ExeCRE enables more reliable use of generated code in execution-based pipelines.

Article Search Article: ase26main-p496-p doi:10.1145/3832783.3837445

DAGGER: Dynamic Agentic Graph-Guided Efficient Reasoner for Container Image Dependency Optimization
Shikhar Vashistha and Neetesh Kumar
(IIT Roorkee, India)
Container image (CI) bloat degrades cloud deployment efficiency, as structural redundancy and obsolete package buildup over time result in complex dependency graphs that remain largely unaddressed by current CI optimization methods. Traditional visualization techniques rely on user expertise for manual optimization, whereas runtime methods require long execution traces to identify and retain only accessed dependencies. Conversely, static-analysis-based methods lack semantic visibility in dynamic loading or reflection contexts, leading to over- or under-approximations. To address these limitations, we propose Dynamic Agentic Graph-Guided Efficient Reasoner (DAGGER) that dynamically selects from an ensemble of optimization strategies based on CI structures and dependency patterns. DAGGER’s agentic AI module utilizes knowledge retrieval on the graph representation of CI dependencies to analyze and prevent the removal of critical dependencies, ensuring zero-loss operational functionality. DAGGER’s knowledge base is built using curated examples that its agentic AI module uses to generate optimized CI. Based on preliminary experiments on production CIs, DAGGER achieves an average size reduction of 35.7% with a 0.2% breakage rate, compared with state-of-the-art baselines.

Article Search Artifacts Available Article: ase26main-p522-p doi:10.1145/3832783.3837446

RamFuzz: LLM-Guided Greybox Fuzzing for Spatial Memory Corruption via Valid Range Violation
Shangzhi Xu, Wei Song, Yuekang Li, Nan Sun, Muhammad Ejaz Ahmed, Willy Susilo, Benjamin Turnbull, Xiao Cheng, and Siqi Ma
(University of New South Wales, Australia; CSIRO's Data61, Australia; University of Wollongong, Australia; Macquarie University, Australia)
Spatial memory corruptions remain among the most prevalent software vulnerabilities, consistently ranking among the top CWE categories. These vulnerabilities are closely tied to the values processed by the program: when input-derived values fall outside expected ranges, memory operations may be performed beyond their allocated bounds, leading to corruption. Our pilot study on 132 realworld CVEs shows that 72.7% of spatial memory corruptions stem from such valid value violations. Although fuzzing is widely used to expose exploitable memory corruptions, systematically detecting spatial memory corruptions through fuzzing remains challenging, as it requires generating inputs that conform to complex structural formats while violating valid value ranges. Recent work leverages LLMs to generate structured inputs based on format knowledge, yet LLMs exhibit a strong bias toward well-formed outputs, leaving systematic range violations largely unexplored. In this paper, we propose RamFuzz, an approach that exploits rather than mitigates this bias to guide the fuzzing process. RamFuzz prompts LLMs to enumerate valid ranges of target file formats and deliberately assigns invalid values to seeds during fuzzing, systematically triggering spatial memory corruptions. We evaluated RamFuzz on 15 popular real-world projects with more than 1,000 stars on GitHub, RamFuzz discovered 104 previously unknown memory corruptions, of which 39 have been confirmed, and 33 have been fixed by developers.

Article Search Article: ase26main-p540-p doi:10.1145/3832783.3837447

To Ban or Not to Ban? How Open Source Projects Govern GenAI Contributions
Wenhao Yang, Runzhi He, and Minghui Zhou
(Peking University, China)
Generative AI (GenAI) is playing an increasingly important role in open source software (OSS). Beyond completing code and documentation, GenAI is increasingly involved in issues, pull requests, code reviews, and security reports. Yet, cheaper generation does not mean cheaper review — and the resulting maintenance burden has pushed OSS projects to experiment with GenAI-specific rules in contribution guidelines, security policies, and repository instructions, even including a total ban on AI-assisted contributions. However, governing GenAI in OSS is far more than a ban-or-not question. The responses remain scattered, with neither a shared governance framework in practice nor a systematic understanding in research. Therefore, in this paper, we conduct a multi-stage analysis on various qualitative materials related to GenAI governance retrieved from 67 highly visible OSS projects. Our analysis identifies recurring concerns across contribution workflows, derives three governance orientations, and maps out 12 governance strategies and their policy instruments. We show that governing GenAI in OSS extends well beyond banning — it requires coordinated responses across accountability, verification, review capacity, code provenance, and platform infrastructure. Overall, our work distills dispersed community practices into a structured overview, providing a conceptual baseline for researchers and a practical reference for maintainers and platform designers.

Article Search Artifacts Available Article: ase26main-p575-p doi:10.1145/3832783.3837448

GraphDroid: Asynchronous LLM-Based Mobile App GUI Testing via History-Aware Exploration and Hybrid Intent Fulfillment
Xiaolei Li, Jialun Cao, Zhijian Hou, Yuzhi Zhao, Yepang Liu, and Shing-Chi Cheung
(Hong Kong University of Science and Technology, China; Southern University of Science and Technology, China; Guangzhou HKUST Fok Ying Tung Research Institute, China; City University of Hong Kong, China)
Automated GUI testing is a widely adopted technique for ensuring mobile application quality by simulating user interactions to exercise functionalities. Despite the research breakthroughs in the past decades, covering complex functionalities that require multi-step action sequences still remains challenging. Traditional tools lack semantic understanding capability and can rarely synthesize such action sequences. Recent LLM-based tools can generate test intents describing target functionalities and leverage the LLM to fulfill the intents, but suffer from three key limitations: 1) loss of historical context for identifying uncovered functionalities, 2) synchronous intent generation that blocks exploration, and 3) per-step LLM-driven fulfillment incurring high cost and latency. To address these limitations, we propose GraphDroid, an intent-driven GUI testing framework that integrates a cluster-based memory mechanism to effectively identify uncovered functionalities from historically visited states for comprehensive application testing. For improving testing efficiency, GraphDroid adopts an asynchronous intent generation paradigm that eliminates the latency bottleneck and a hybrid intent fulfillment strategy that reserves the LLM for fulfilling complex intents while delegating simple intents to a lightweight heuristic algorithm. We evaluate GraphDroid on 41 real-world Android apps against six state-of-the-art baselines. Results show that GraphDroid outperforms all baselines, achieving up to 36.4% higher code coverage while incurring less than one eighth of the cost of the best pure LLM-based baseline. GraphDroid also exposes 19 bugs in the 41 apps and detects 13 of 52 crashes in the Themis bug benchmark, surpassing all the six baselines. Seven of the 19 bugs were previously unknown and we reported them to the developers. So far, four bugs have been confirmed and fixed.

Article Search Article: ase26main-p609-p doi:10.1145/3832783.3837449

VARIES: Verification Harness Synthesis and Efficient Scheduling for Unsoundness Detection in Rust Libraries
Huan Li, Xing Hu, Xin Xia, and Xinyu Wang
(Zhejiang University, China)
Rust provides strong compile-time safety guarantees, but unsafe code in real-world libraries can still introduce unsoundness bugs that may undermine this guarantee. Compared with prior work, bounded model checking (BMC) offers a promising direction by adapting formal verification to unsoundness detection. In practice, however, BMC requires harnesses that are tedious to construct, and many unsoundness bugs arise only under specific program states that a single harness may miss. To this end, we propose Varies, a BMC-based framework that treats unsoundness detection in Rust libraries as a joint problem of harness synthesis and budget-aware verification. During the synthesis stage, Varies builds seed harnesses for each target API and then expands them into derived harnesses through mutator injection. Over the resulting harness derivation tree, Varies applies feedback-driven pruning strategies to schedule verification efficiently. In our benchmark, Varies identifies defects involving target APIs in more evaluated crates than any baseline while maintaining a low false-positive rate. In a broader deployment on popular crates.io libraries, it identifies previously unknown unsoundness in 10 crates, with reports for 6 crates acknowledged by maintainers. The results indicate that stateful harness synthesis and feedback-driven scheduling can make BMC practical for targeted unsoundness detection in Rust libraries.

Article Search Article: ase26main-p638-p doi:10.1145/3832783.3837450

Prism: A Multi-solution Reasoning and Synthesis Framework for Repository-Level Issue Resolution
Yibo Wang, Guotian Wang, Zhihao Peng, Ying Wang, Chang Xu, and Zhiliang Zhu
(Northeastern University, China; Nanjing University, China)
Large language model (LLM)-based agents have achieved notable success in repository-level issue resolution, yet existing frameworks still face two key bottlenecks: insufficient diversity among candidate repair solutions and limited ability to combine complementary strengths across candidate solutions. To address these challenges, we introduce Prism, a multi-solution reasoning and synthesis framework that adopts a coarse-to-fine paradigm to systematically generate, refine, and integrate diverse repair solutions. Prism comprises three stages: (1) a global exploration mechanism with contrastive semantic constraints that broadens the search space by encouraging divergence across historical solutions; (2) an A* algorithm-inspired local exploration mechanism that identifies suitable branching points to generate divergent sub-solutions; and (3) a multi-agent collaborative solution synthesis process where reviewer and judge agents jointly evaluate, complement, and integrate partially correct candidate solutions. We evaluate Prism on SWE-bench Verified and SWE-bench-Live Lite. With Claude-4.6-Sonnet and DeepSeek-V3.2-Reasoner, Prism achieves Pass@1 rates of 82.0% and 80.0% on SWE-bench Verified, respectively, outperforming the compared baselines. On SWE-bench-Live Lite, designed to reduce data leakage concerns, Prism outperforms the reported baselines. Further analyses show that Prism expands the solution space and that its solution synthesis stage uniquely resolves 15 issues unaddressed by isolated candidates.

Article Search Article: ase26main-p663-p doi:10.1145/3832783.3837451

RepoProbe: Benchmarking Architecture-Aware Repository Comprehension with Checklists
Yuexi Yang, Alyssa Wu, Ji Luo, Richeng Xuan, Zhichao Hu, Yuhong Liu, and Zhen Qin
(Zhejiang University, China; Tencent, China)
The integration of Large Language Models (LLMs) into software engineering has shifted the focus from function-level generation to repository-scale assistance. However, existing benchmarks largely rely on bug reports from GitHub Issues, which often allow models to bypass genuine understanding via pattern matching on error logs. This misalignment under-measures Edit Bias, which refers to premature generation, where models prematurely propose code modifications instead of understanding the existing repository architecture. Furthermore, current LLM-as-a-Judge scalar scoring suffers from high variance and low interpretability. This work introduces RepoProbe, a novel benchmark for evaluating repository-level code understanding through open-ended Q&A using GitHub Discussions, which focuses on open-ended architectural inquiries rather than defect reporting. To ensure rigorous evaluation, we propose a Checklist-Based Verification Protocol that decomposes answers into atomic, verifiable facts, thereby replacing subjective ratings with objective verification. Our evaluation of state-of-the-art (SOTA) LLMs reveals a persistent gap between high clarity and evidence-grounded technical correctness. It also quantitatively confirms the prevalence of edit bias, in which models prioritize code generation instead of architectural analysis. Finally, we demonstrate that our verification protocol significantly improves evaluation reliability compared to traditional evaluations with scalar scoring.

Article Search Artifacts Available Article: ase26main-p676-p doi:10.1145/3832783.3837452

RFC2TLA+: Extracting and Verifying Formal Models from RFC Documents using Continuous LLM Feedback
Guozhen Ding, Kexin Li, Ilya Grishchenko, and David Lie
(University of Toronto, Canada)
Extracting natural-language protocol specifications such as Requests for Comments (RFCs) into formally verifiable models is a challenging and labor-intensive process that requires significant expertise. Recent advances in large language models (LLMs) and agents show promise in automating this task. However, existing automated approaches suffer from insufficient structural consistency, hallucinations, and limited scalability. Moreover, the ability to pass such automatically extracted models into formal verification tools such as model checkers has not been demonstrated.
We present RFC2TLA+, a three-stage pipeline for automatically extracting RFCs into automatically verifiable models. RFC2TLA+ achieves this by providing fine-grain and continuous feedback to the LLM throughout the extraction process. It decomposes the extraction process into three passes—states, transitions, and invariants—while constraining LLM generation through fine-grained tool calls, lightweight validation, and iterative feedback. This design enables early error detection and correction, mitigating hallucinations and preventing syntactic and semantic inconsistencies. The resulting communicating finite-state machines (CFSMs) are deterministically translated into TLA + and verified using the TLC model checker.
We evaluate RFC2TLA+ on a benchmark of 14 RFCs and their accompanying human-crafted FSMs and show that it generates more complete models than the baseline. In addition, our tool extracts invariants, enabling a model checker to detect injected faults. To minimize the effects of LLM memorization, we additionally generate synthetic RFCs and evaluate the tool on them, demonstrating its effectiveness in both model extraction and error detection.

Article Search Artifacts Available Article: ase26main-p683-p doi:10.1145/3832783.3837453

It Takes Two to Tango: Accelerating Code Generation via Cache-Inspired Hierarchical Speculative Decoding
Yinggang Qiu, Shangwen Wang, Yihao Qin, Mingyang Geng, and Dezun Dong
(National University of Defense Technology, China; Information Support Force Engineering University, China)
While Large Language Models (LLMs) have demonstrated exceptional performance in code generation, their autoregressive decoding paradigm typically results in low inference efficiency. Speculative decoding has emerged as a solution to speed up inference through its draft–verification mechanism, where multiple draft tokens are generated efficiently and then verified in parallel by LLMs. Current speculative decoding approaches for code generation are broadly categorized into two types: model-based approaches that generate drafts using a lightweight draft model, and retrieval-based approaches that reuse code snippets from external corpora. In this work, we first conduct empirical experiments to analyze the strengths and weaknesses of retrieval-based and model-based drafting. We find that retrieval-based approach is efficient but usually low-quality, while model-based one is high-quality but the drafting time increases with inference steps of the draft model. Inspired by the cache mechanism in computer architecture, we propose a novel framework CacheCoder. In our design, retrieval acts as a high-speed Level-1 (L1) cache for rapid draft acquisition, while the draft model functions as a more capable Level-2 (L2) cache to compensate for L1 misses, enabling us to capitalize on the speed of reuse when possible and rely on the precision of generation when necessary. To combine these two hierarchical processes, we propose two customized strategies, i.e., Preliminary Prediction and Draft Expansion, which utilize the draft model to pre-filter retrieved contents and expand them with tokens generated by the draft model to form drafts with higher length. Experimental results show that CacheCoder achieves up to a 2.64× speedup compared to autoregressive decoding, significantly outperforming state-of-the-art approaches.

Article Search Article: ase26main-p691-p doi:10.1145/3832783.3837454

One Is Not Enough: The Untold Story of Multiple Security Patches for One Vulnerability
Fangyuan Zhang, Lyuye Zhang, Lingling Fan, Chengwei Liu, Yinan Li, Liang Huang, Yang Liu, Zheli Liu, and Sen Chen
(Nankai University, China; Nanyang Technological University, Singapore; Qi An Xin Technology Group, China)
Security patches (SPs) are the main mechanism for fixing software vulnerabilities, yet a single vulnerability is not always resolved by a single patch: fixes may be completed incrementally, propagated across maintained branches, or replicated across related repositories. When patch records are incomplete, downstream users may observe only part of the required fix set and therefore apply only partial patching. However, comprehensive patch discovery remains difficult because the prevalence and causes of the multi-SP phenomenon are still poorly understood. In this paper, we present the first large-scale empirical study of multi-SP vulnerabilities. By merging four major vulnerability databases, we construct a dataset of 6,053 multi-SP CVEs with 16,260 SPs, showing that 20.6% of CVEs with patches involve multiple SPs and that merging databases increases recognized multi-SP CVE counts by 36-55% over any single source. We further analyze why a vulnerability is associated with multiple SPs and derive a two-level taxonomy with 6 categories and 16 sub-categories. Based on these findings, we develop SPectre, a taxonomy-driven prototype for comprehensive patch discovery. On 300 multi-SP CVEs, after manually verifying ground-truth SPs, SPectre improves multi-SP patch coverage over representative patch-localization baselines, achieving 0.927 recall on same-repository cases and 0.873 recall on cross-repository cases after manual ground-truth verification. On 100 recent CVEs recorded as single-patch by all public databases, SPectre further discovers 28 previously unreported SPs across 20 CVEs. Our results show that multi-SP vulnerabilities are both prevalent and systematically underreported, motivating stronger patch-completeness awareness, improved vulnerability database curation, and relation-aware security tooling.

Article Search Article: ase26main-p694-p doi:10.1145/3832783.3837455

Not as Sweet by Another Name: An Empirical Study of Format Robustness in LLM Document Workflows
Xiaoyu Zhang, Xianyun Cheng, Tianlin Li, Yuwei Zheng, Yue Yang, and Yang Liu
(Nanyang Technological University, Singapore; National University of Singapore, Singapore; Beihang University, China; Xi'an University of Architecture and Technology, China)
LLM-driven software systems are rapidly evolving from plain-text conversations to document-centric end-to-end workflows, where the same semantic content can be delivered in diverse document formats (e.g., CSV) through file upload interfaces. Yet existing testing work focuses on the robustness and reliability of models and systems whose input is a single prompt string, leaving a critical question unanswered: Can these document workflows maintain robust behaviors when the same content arrives in a different document format? To fill the gap, in this paper, we propose a format-aware metamorphic testing framework with three metamorphic relations to comprehensively evaluate the format robustness of end-to-end LLM document workflows. Based on this framework, we conduct a large-scale empirical study spanning four representative LLM workflows, four real-world tasks, and four document formats, comprising a total of 48,000 workflow executions. Our findings reveal that format variation poses a systematic and serious threat. Merely switching formats can cause accuracy to drop by up to 53.63% and trigger decision drifts in over 41% of instances. We further design lightweight mitigation strategies from the users’ perspective that recover up to 44.21% of format-induced decision drift without model retraining. Our study demonstrates that document format is not a neutral wrapper but a critical factor affecting the reliability of LLM software systems, calling for corresponding testing and safeguards in the deployment in real-world high-stakes scenarios.

Article Search Article: ase26main-p718-p doi:10.1145/3832783.3837456

RealisticTritonBench: A Benchmark for Triton-Kernel Generation in Real-World AI Frameworks
Jinjun Huang, Zhongzhen Wen, Tongtong Xu, Meng Yan, Xin Xia, and Zhongxin Liu
(Zhejiang University, China; Nanjing University, China; Huawei, China; Chongqing University, China)
In modern AI frameworks, the GPU kernel is a key determinant of overall system performance. By combining usability, portability, and near-handwritten CUDA performance, Triton has been widely adopted for implementing GPU kernels. Recent advances have shown the potential of using large language models (LLMs) to automatically generate Triton kernels, helping reduce the manual effort required by expert kernel developers. To evaluate the quality of LLM-generated Triton kernels, several benchmarks have been proposed. However, existing benchmarks primarily target isolated kernel generation tasks and suffer from three key limitations: (1) they restrict the task to only PyTorch-to-Triton translation, thus failing to reflect the diversity and complexity of real-world Triton tasks; (2) they evaluate only the performance of individual kernels, limiting the evaluation of real-world performance of generated kernels in AI frameworks, where end-to-end performance is the core criterion for real deployment; (3) they rely on manually written evaluation scripts for single kernel, which may introduce vulnerabilities and allow models to exploit evaluation flaws to bypass correctness checks and obtain inflated scores. To address these limitations, we introduce RealisticTritonBench, the first benchmark that derives Triton kernel generation tasks from real-world pull requests in popular AI frameworks, enabling evaluation under realistic, production-like settings. RealisticTritonBench systematically extracts real-world PRs with Triton kernel modified from popular open-source AI frameworks and transforms them into kernel generation tasks with concrete engineering contexts. Each task takes a natural language requirement as input and requires to generate a corresponding Triton kernel implementation. RealisticTritonBench also provides a complete and reproducible evaluation environment for each task. In contrast to prior benchmarks that focus on isolated kernel performance, RealisticTritonBench integrates generated kernels into their original frameworks and evaluates them under end-to-end test, enabling a more faithful assessment. We conduct a systematic evaluation of the state-of-the-art LLMs on RealisticTritonBench, revealing that they still struggle to handle the challenges in real-world Triton generation tasks.

Article Search Article: ase26main-p721-p doi:10.1145/3832783.3837457

From Guessing to Seeing: Enhancing LLM-Based Program Repair via Trace-Guided Multi-strategy Debate
Jiaqing Wu, Tong Wu, Manqing Zhang, Yunwei Dong, and Bo Shen
(Northwestern Polytechnical University, China)
Automated Program Repair (APR) aims to resolve software bugs without human intervention. However, handling complex logic errors, especially silent failures that produce incorrect outputs without any explicit crash signals, remains a significant challenge. Current LLM-based APR approaches attempt to address this issue through conversational feedback, retrieval-augmented generation, or static analysis tools. However, these methods are fundamentally static, relying only on source code and basic test outputs, and thus struggle to capture complex runtime behaviors and dynamic data dependencies accurately. Inspired by human debugging practice, incorporating runtime evidence into APR provides direct access to program behavior through execution traces, which expose concrete state transitions and data dependencies. However, even with such runtime evidence, a single LLM interpreting it in isolation tends to commit to a specific repair hypothesis, leading to test overfitting and producing patches that satisfy the observed traces and test suite by coincidence rather than correct logic. This observation suggests that runtime evidence should not be treated merely as additional input, but as objective constraints that candidate patches must satisfy. Building on this insight, we propose TraceRepair, a multi-agent framework that leverages runtime facts as shared constraints for patch validation. A probe agent captures execution snapshots of critical variables, forming an objective basis for repair, while a committee of specialized agents cross-verifies candidate patches to expose inconsistencies and iteratively refine them. Evaluated on the Defects4J benchmark, TraceRepair substantially improves repair effectiveness, correctly fixing 392 defects and outperforming existing LLM-based approaches. Extensive experiments further demonstrate improved efficiency and strong generalization on a newly constructed dataset of recent bugs, suggesting that the performance gains arise from dynamic reasoning rather than memorization.

Article Search Article: ase26main-p732-p doi:10.1145/3832783.3837458

When Extensions Lie: Large-Scale Study of Privacy Policy Compliance in Browser Extensions
Zilun Wang, Zeyang Zhuang, Mingxue Zhang, Wei Meng, and Michael R. Lyu
(Chinese University of Hong Kong, China; Zhejiang University, China; Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security, China)
Browser extensions can pose significant privacy risks to end users due to their complex permission mechanisms and high privilege levels. Modern websites increasingly embed sensitive personal data within HTML elements, making such threats even more insidious. Although prior studies have examined browser extension privacy risks from different perspectives, they are typically constrained to specific sensitive data sources or a set of extensions tested on a few popular websites.
To enable a comprehensive compliance check of browser extensions' privacy practices at scale, we develop an automated dynamic analysis framework, ExtPolice. It applies LLM-based techniques to identify sensitive web content embedded in web pages, performs a multi-type taint analysis in the browser to comprehensively track sensitive data flows across browser components, and checks the compliance of such flows against the extension's privacy policy disclosures. We apply ExtPolice to evaluate 149,216 Chrome extensions across 7,200 unique web pages and identify 2,246 extensions with policy violations, enabling scalable analysis of policy compliance for browser extensions. Our study uncovers thousands of policy violations potentially affecting up to tens of millions of users, highlighting major gaps in current extension-store review processes.

Article Search Article: ase26main-p741-p doi:10.1145/3832783.3837459

MultiKernelBench: A Multi-platform Benchmark for Kernel Generation
Zhongzhen Wen, Yinghui Zhang, Zhong Li, Zhongxin Liu, Linna Xie, and Tian Zhang
(Nanjing University, China; Zhejiang University, China)
The automatic generation of deep learning (DL) kernels using large language models (LLMs) has emerged as a promising approach to reduce the manual effort and hardware-specific expertise required for writing high-performance operator implementations. However, existing benchmarks for evaluating LLMs in this domain suffer from limited hardware support, coarse-grained kernel categorization, and imbalanced task coverage. To address these limitations, we introduce MultiKernelBench, the first comprehensive, multi-platform benchmark for LLM-based DL kernel generation. MultiKernelBench spans 300 tasks across 15 well-defined kernel categories and supports three major hardware platforms: NVIDIA GPUs, Huawei NPUs, and Google TPUs. To enable future extensibility, we design a modular backend abstraction layer that decouples platform-specific logic from the core benchmarking infrastructure, allowing easy integration of new hardware platforms. We further propose a simple yet effective category-aware one-shot prompting method that improves generation quality by providing in-category exemplars. Through systematic evaluations of seven state-of-the-art LLMs, we reveal significant variation in task difficulty, poor generalization to platforms with less training exposure, and the effectiveness of targeted prompting strategies.

Article Search Article: ase26main-p755-p doi:10.1145/3832783.3837460

Post-quantum Cryptography in the Wild: Assessing the Readiness of Open-Source Ecosystems
Tongxin Yuan, Zhanpeng Liu, Jiashuo Liang, Zhuosheng Zhang, Gongshen Liu, Yang Yu, and Guancheng Li
(Shanghai Jiao Tong University, China; Tencent Xuanwu Lab, China)
The threat of cryptographically relevant quantum computers, compounded by harvest now, decrypt later attacks, has made migration to post-quantum cryptography (PQC) urgent. With PQC standards finalized and transition guidance phasing out quantum-vulnerable algorithms, the readiness of the open-source supply chain, through which cryptographic vulnerabilities propagate, remains unclear. To fill this gap, we present the first cross-ecosystem empirical study of source-level PQC readiness. PQC Scanner is a two-stage pipeline in which (i) crypto detection identifies packages with cryptographic functionality, and (ii) PQC detection produces a binary support label and source-derived structured readiness record at the repository level using commit history, source analysis, and agent post-verification. Applied to 1,117,913 packages, Stage 1 identifies 55,654 crypto-related packages. Stage 2 evaluates their repositories for source-level PQC support. A separate audit of 219 Stage 2-negative repositories gives a Stage 2 recall lower bound of
≥
 97.48%. On a 247-repository deployment-validation sample, agent post-verification eliminated all false positives, yielding precision of 1.000. The pipeline scales to million-package scans: at concurrency 10, it analyzes over one million packages in just over two weeks, with a median LLM cost of $0.0010 per package. Across 8 ecosystems, we find a migration gap: only 1.88% of crypto-related packages show source-level PQC support, and the repository-deduplicated rate is 1.72%. Rust leads by both counts, while NuGet/Ruby/PHP remain near zero. Source-derived structured readiness records show partial support: among PQC-supporting repositories, only 19.2% satisfy the full-support criterion, with a mean migration coverage of 36%. Adoption accelerates sharply after FIPS publication (August 2024), but migration remains infrastructure-gated: key libraries such as Python’s cryptography (581M downloads/month) and framework integration layers in Java/npm still lack deployable PQC pathways, blocking downstream packages regardless of developer intent. We characterize adoption timing, bottlenecks, and community response strategies, deriving recommendations for maintainers, registry operators, and policy makers.

Article Search Article: ase26main-p758-p doi:10.1145/3832783.3837461

CodeFault: Predicting Fault Risks from Code Changes via Multi-modal Learning
Yifan Xiao, Shijie Li, and Yu Huang
(Peking University, China; China Southern Power Grid, China)
Just-In-Time defect prediction identifies fault-inducing code changes but relies solely on static code properties, missing runtime fault patterns. Meanwhile, log-based anomaly detection captures runtime behaviors but lacks visibility into causal code modifications. This disconnect delays fault identification and complicates root cause analysis. We present CodeFault, a multi-modal fault prediction framework that fuses code change semantics with runtime log patterns through two novel mechanisms. Hierarchical Fault Spectrum Alignment (HFSA) projects both modalities into a shared, two-level fault taxonomy space—four coarse and eight fine-grained categories—providing a structured information bottleneck that enables principled cross-modal knowledge transfer. Spectrum-Guided Cross-Modal Attention (SGCA) performs token-level fusion conditioned on fault spectrum agreement, avoiding the degeneracy of pooled-vector approaches while focusing attention on fault-relevant token pairs. A dual-granularity contrastive objective further leverages the taxonomy structure to separate fault-inducing from benign embeddings at both instance and fault-type levels. We evaluate CodeFault on two complementary testbeds. On the public benchmark of 31,142 samples from Defects4J, BugsInPy, and ManySStuBs4J, CodeFault reaches 0.891 AUC-ROC and 0.857 F1-score against 15 prior-art baselines and 2 self-constructed strong late-fusion controls, improving over the strongest code-only baseline by 5.1% AUC and over the strongest late-fusion control by 3.1% AUC (Wilcoxon signed-rank test, Cliff's δ ≥ 0.73). On 2,147 real co-occurring CI/CD pairs collected from 55 GitHub Actions repositories with the CI build outcome as ground truth—the largest non-synthetic CI/CD fault test set in JIT defect prediction to date—CodeFault reaches 0.843 AUC, +4.3% over JIT-Smart, retaining 94.6% of the synthetic-set headline. The framework generalizes to unseen projects with 2.5% average AUC improvement and maintains 97.9% AUC under realistic 15% positive rates. A knowledge-distilled code-only variant achieves 0.862/0.831 AUC on synthetic/real test sets without log inputs, enabling pre-commit deployment.

Article Search Artifacts Available Article: ase26main-p759-p doi:10.1145/3832783.3837462

ReLog: Execution-Aware Logging with Runtime Feedback for LLM-Oriented Debugging
Xin Wang, Yang Feng, Xiaoqian Jiao, Yang Zhang, Zhenhao Li, and Zishuo Ding
(Hong Kong University of Science and Technology, Guangzhou, China; Hebei University of Science and Technology, China; York University, Canada)
Logging statements are essential for software debugging and maintenance. However, existing approaches to automatic logging generation rely on static analysis and produce statements in a single pass without considering runtime behavior. They are also typically evaluated by similarity to developer-written logs, assuming these logs form an adequate gold standard. This assumption is increasingly limiting in the LLM era, where logs are consumed not only by developers but also by LLMs for downstream tasks. As a result, optimizing logs for human similarity does not necessarily reflect their practical utility. To address these limitations, we introduce ReLog, an iterative logging generation framework guided by runtime feedback. ReLog leverages LLMs to generate, execute, evaluate, and refine logging statements so that runtime logs better support downstream tasks. Instead of comparing against developer-written logs, we evaluate ReLog through downstream debugging tasks, including defect localization and repair. We construct a benchmark based on Defects4J under both direct and indirect debugging settings. Results show that ReLog consistently outperforms all baselines, achieving an F1 score of 0.520 and repairing 97 defects in the direct setting, and the best F1 score of 0.408 in the indirect setting where source code is unavailable. Additional experiments across multiple LLMs demonstrate the generality of the framework, while ablations confirm the importance of iterative refinement and compilation repair. Overall, our work reframes logging as a runtime-guided, task-oriented process and advocates evaluating logs by their downstream utility rather than textual similarity.

Article Search Article: ase26main-p769-p doi:10.1145/3832783.3837463

Documentation vs. Code Patterns: What Drives LLM-Based Exception Oracle Generation?
Soneya Binta Hossain, Matthew B. Dwyer, and Tasfia Tasnim
(University of Texas at Dallas, USA; University of Virginia, USA)
LLM-based test oracle generation (TOG) methods report high accuracy on exception-oracle generation, but it remains unclear what evidence drives these predictions. In particular, do models use explicit exceptional-behavior documentation such as Javadoc @throws clauses, or do they rely on recurring patterns in tests, code, and documentation?
We investigate this question through a large-scale intervention-based study of three TOG systems spanning classifier-based and generative architectures and model sizes from roughly 110M to 7B parameters, evaluated on three real-world benchmarks comprising two generated-test datasets and a new benchmark of developer-written tests. We first remove Javadoc @throws clauses and find that accuracy changes only marginally, with the largest drop below one percentage point. This indicates that structured exception documentation is not the primary driver of exception-oracle prediction. We then apply attribution-guided substitution ablations to identify the signals that predictions depend on. The results show that high accuracy can be driven by shortcut signals: some models are highly sensitive to a small number of structural tokens, while others distribute reliance across many lexical cues.
These findings challenge the assumption that strong exception-oracle accuracy reflects robust use of exception semantics. Future TOG systems should therefore be evaluated not only by whether they predict the correct oracle type, but also by whether their predictions are grounded in meaningful exception-triggering evidence.

Article Search Article: ase26main-p773-p doi:10.1145/3832783.3837464

Harnessing Uncertainty in Code Language Models: Lessons from Vulnerability Detection
Haodong Li, Xiao Cheng, Xudong Wang, Zhihao Guo, and Haoyu Wang
(Huazhong University of Science and Technology, China; Macquarie University, Australia; Zhongguancun Institute of Artificial Intelligence, China; University of Technology Sydney, Australia)
Code Language Models (CLMs) are increasingly adopted for vulnerability detection, yet knowing when to trust their predictions remains an open challenge. Uncertainty estimation has improved reliability in traditional discriminative models, but whether these benefits transfer to the generative paradigm of CLMs is unclear. This experience paper presents a systematic empirical study of uncertainty estimation in CLM-based vulnerability detection, evaluating four calibration methods and two uncertainty-driven enhancement frameworks across eight models (2B--16B parameters) under both in-distribution and out-of-distribution settings on 1,050 real-world projects. Our study reveals that: (1) calibration effectiveness is model-scale dependent: temperature scaling and ensembles deliver consistent gains, while Monte Carlo Dropout and Bayesian Low-Rank Adaptation by Backpropagation only benefit medium-scale models (approximately 6B--7B parameters); (2) uncertainty-based correction requires label-conditional modeling to be effective (up to 12% Macro-F1 improvement on out-of-distribution data), as global strategies show negligible effects; (3) curriculum learning fails to improve upon standard fine-tuning, as uncertainty-based difficulty ordering does not align with the learned decision boundary, causing training instability or negligible gains. These findings challenge assumptions about the universal transferability of uncertainty methods and provide practical guidelines for deploying uncertainty-aware vulnerability detection.

Article Search Article: ase26main-p786-p doi:10.1145/3832783.3837465

Breaking the Isolation: Coordinated Multi-agent Fuzzing for Smart Contracts with Multi-dimensional Objective Learning
Ruichao Liang, Ruochen Cao, Jing Chen, Kun He, Yebo Feng, Yue Xue, and Cong Wu
(Wuhan University, China; Nanyang Technological University, Singapore; MetaTrust Labs, Singapore)
Smart contracts manage critical assets in decentralized systems but are prone to vulnerabilities. While fuzzing has proven effective in software testing, existing contract fuzzers often rely on ad-hoc optimization strategies with limited objective guidance, typically driven by coverage or simple heuristics. This narrow focus leads to high false negative rates, as important perspectives of smart contracts, like semantic, state, and economic effects, are overlooked during transaction sequence generation. Moreover, coordinating these dimensions into fuzzing is challenging because their divergent objectives and complex interactions make ad-hoc adaptations to fitness functions or mutation strategies largely ineffective.
To fill the gaps, we propose SynerFuzz, a multi-agent deep reinforcement learning-based fuzzer. We formulate fuzzing as a Decentralized Partially Observable Markov Decision Process, where each agent is assigned a distinct testing objective extracted from the target contract. Through coordinated learning, DRL agents integrate multi-dimensional signals to construct high-quality sequences, enabling SynerFuzz to uncover both subtle semantic flaws and economically profitable behaviors. Compared to 9 SOTA tools, SynerFuzz detects up to 4.2× more vulnerabilities while reducing time and transaction overhead by 70.6% and 86.8%. In real-world scenarios, SynerFuzz reproduces 33 historical DeFi exploits with over $40 million losses and discovers 17 0-day vulnerabilities in 4 projects managing over $179.8 million, earning $3,800 bug bounties.

Article Search Article: ase26main-p791-p doi:10.1145/3832783.3837466

HyTri: Hybrid Triage of CI Failures via LLM Guided Semantic Reasoning and Change Attribution
Lior Broide, Argaman Mordoch, and Roni Stern
(Ben-Gurion University of the Negev, Israel)
Continuous Integration (CI) builds execute large test suites that often yield multiple simultaneous failures, many originating from a common underlying fault. Test failure triage -- grouping failing tests by their root cause -- is therefore critical for reducing debugging effort. Existing approaches work poorly in CI builds with multiple faults, and either lack semantic reasoning or ignore highly informative structural and change-aware signals. To address these limitations, we propose HyTri, a framework that integrates LLM-guided causal reasoning into test failure triage. HyTri derives similarity signals that consider failure semantics and hypothesized root causes. It fuses these signals using confidence weighting and error-type adjustment into a unified similarity measure that is used to cluster the failing tests. To evaluate HyTri we created Multi-Fault CI Build Benchmark (MFCI-Benchmark), a novel benchmark for test failure triage algorithms comprising synthesized multi-fault CI builds from well-known bug repositories. Experimental results over MFCI-Benchmark and the multifault Defects4J fault localization benchmark showed that HyTri outperforms existing baselines, maintaining strong performance even in datasets where stack traces are shallow and uninformative.

Article Search Article: ase26main-p811-p doi:10.1145/3832783.3837467

Latent Reuse in Agent Skills: Multi-modal Clone Detection at Ecosystem Scale
Jiaying Zhu, Lyuye Zhang, Wenbo Guo, and Yang Liu
(Nanyang Technological University, Singapore)
An agent skill is a reusable package for extending an LLM agent, typically a SKILL.md file that combines YAML metadata, natural-language instructions, and executable code. Public repositories now host over two million skills, yet existing tools analyze each artifact in isolation, and registries do not track reuse created through copying, renaming, or adaptation. Detecting these links is difficult because reuse may appear in one channel while the others change; conventional single-channel clone detectors can therefore miss such adaptations. We present SkillReuse, a multi-modal clone detector that combines global lexical matching with channel-specific representations for YAML, prose, and code, then uses logistic regression to produce clone scores and interpretable clone-type labels. We also introduce SkillReuse-Bench, an annotated benchmark of 300 skill pairs spanning exact copies, renamed copies, adaptations, and semantic equivalents. On SkillReuse-Bench, SkillReuse reaches an F1 of 0.939 with 0.952 precision, improving over TF-IDF and delivering 4.2× higher recall on Type-4 semantic clones than MinHash. Applied to all 137,470 skills that pass the content filter, SkillReuse identifies 1.06 million clone pairs involving 66.8% of the analyzed skills; 95.3% of these pairs cross author boundaries. Among skills in the analyzed name-based clone families, 67% are superseded by a higher-quality variant. Tracing 938 security-relevant skills through the clone graph surfaces 16,587 clone links spanning 6,376 related skills that per-skill scanners alone would miss.

Article Search Article: ase26main-p820-p doi:10.1145/3832783.3837468

Specification-Guided Synthesis of Deadlock-Free Communication Protocol Refinements with Large Language Models
Yang Li, Ping Hou, and Nobuko Yoshida
(University of Oxford, UK)
Ensuring behavioural correctness in communication protocols is a central challenge in distributed software systems, as subtle inconsistencies can lead to deadlocks. In such settings, protocol refinement – the safe substitution of a protocol that preserves correctness and compatibility with other components – is essential. Large language models (LLMs) have demonstrated strong capabilities in code generation and program synthesis, yet lack mechanisms to reliably produce outputs with correct behaviour. Formal specification approaches, such as multiparty session types (MPST), offer rigorous guarantees, including deadlock freedom, but provide limited support for automatically constructing protocol refinements. In this paper, we present Syntropy, a framework for synthesising protocol refinements guided by MPST specifications and LLMs. It incorporates refinement constraints directly into the generation process, ensuring the generated variants satisfy these guarantees. Our comprehensive evaluation indicates that Syntropy achieves 95.6%–99.5% validity while maintaining high syntactic correctness, and produces diverse, non-trivial refinements across multiple LLMs.

Article Search Article: ase26main-p825-p doi:10.1145/3832783.3837469

Execution-as-Configuration: Security Smells in Model Configuration Artifacts
Mohammed Latif Siddiq, Prince Noah Johnson, and Joanna C. S. Santos
(University of Notre Dame, USA)
Model sharing hubs rely on configuration artifacts to control model loading and execution. Although treated as metadata, files such as config.json determine class resolution and dependency selection, effectively making configurations an execution control surface. Yet their security implications remain largely unexplored. In this paper, we present a large-scale study of model configuration-related security smells. We introduce a taxonomy of 12 security smell types capturing execution-enabling constructs, trust boundary confusion, supply-chain risks, information leakage, and operational risks in model configuration files. To operationalize this taxonomy, we design MONTICELLO, a repository-aware security smell detection approach that combines pattern-based rules with contextual reasoning. We analyze 44,812 repositories across Hugging Face, OpenCSG, and ModelScope. Configuration smells are pervasive: 65.5

Article Search Article: ase26main-p830-p doi:10.1145/3832783.3837470

Evidence-Grounded Logic Vulnerability Detection for Smart Contracts via Runtime Context Reasoning
Guang Gong, Yucong Jiang, Rong Jian, Xiaotao Feng, Zhihan Wei, Qi Li, and Ke Xu
(Tsinghua University, China; 360 Vulnerability Research Institute, China)
Smart contracts are foundational to DeFi, yet detecting business-logic vulnerabilities, i.e., execution is syntactically correct but violates a contract’s intended behavior, remains difficult. Most recent LLM-based approaches still follow a largely static, source-code–centric auditing paradigm. In practice, however, smart contracts are inherently stateful: many logic flaws emerge only under specific contract states and across particular state transitions. As a result, code-only auditing can produce false positives and miss true vulnerabilities. To address this gap, we present RcAudit, a system for detecting smart-contract logic vulnerabilities via runtime-context–enhanced LLM reasoning. RcAudit constructs a Contract Logic Knowledge Base that captures contract behavior from multiple perspectives to support LLM analysis. It then applies a state- and intent-aware fuzzing strategy to generate logic-critical runtime contexts, and performs LLM reasoning over selected contexts that exhibit potential intent–behavior inconsistencies to pinpoint vulnerabilities. We evaluate RcAudit on three real-world datasets. RcAudit identifies 74 of 132 known logic vulnerabilities with high precision (96.9%), achieving 117.6% higher detection than the strongest baseline. Importantly, RcAudit further uncovers 22 zero-day vulnerabilities in real-world projects.

Article Search Artifacts Available Article: ase26main-p839-p doi:10.1145/3832783.3837471

Fully Automating Template Polyhedral Analysis by Leveraging LLMs
Renjie Huang, Liqian Chen, Hongfei Fu, Banghu Yin, Dengping Wei, and Ji Wang
(National University of Defense Technology, China; Shanghai University of Finance and Economics, China)
Abstract interpretation provides a generic framework to design static analysis, and has been widely applied in analysis of real-world programs. The scalability and precision of these analyses depend largely on the chosen abstract domain. The template polyhedra domain is well-known for offering a compelling trade-off between scalability and precision. However, designing effective templates currently requires intensive manual effort and domain expertise. In this paper, we propose a bottom-up approach based on Large Language Models (LLMs) to automatically generate linear templates for the template polyhedral analysis. By leveraging the advanced code comprehension capabilities of LLMs, our approach automatically generates highly relevant templates. Based on this approach, we have implemented IKOSplus, a program analyzer supporting fully automated context-sensitive and interprocedural template polyhedral analysis, on top of IKOS. We evaluated IKOSplus on a variety of small-scale benchmark programs and complex real-world programs. Experimental results demonstrate that IKOSplus effectively balances analytical precision and scalability in the template polyhedral analysis.

Article Search Artifacts Available Article: ase26main-p849-p doi:10.1145/3832783.3837472

When Does AI Actually Help in Incident Response? Identifying Good First Messages in Cloud Service Incidents
Minghua Ma, Rujia Wang, Chetan Bansal, Saravan Rajmohan, Yingnong Dang, and Hongyu Zhang
(Microsoft, USA; Chongqing University, China)
Modern cloud services increasingly rely on agentic software operations, where AI agents, rule-based automation (monitoring alerts, health checks, template runbooks), and human engineers coordinate incident response through shared discussion threads. As these systems mature, the on-call engineer faces a growing volume of machine-generated messages and must quickly identify which ones actually help, under minutes-level time pressure where every moment spent reading noise delays mitigation.
We study this problem across thousands of production incidents from 5 cloud services over four years (2022 to 2026). We define a good first message (GFM): the first machine-generated message in an incident discussion thread whose specific content demonstrably influences the on-call engineer’s next investigation action. Three findings emerge. First, incident threads grew 7× longer (from a median of 4 to 30 messages) as AI agents were introduced, yet noise (messages lacking substantive technical content) decreased: among the three actor types, AI agents, rule-based automation, and humans, AI produces the lowest noise rate. Second, AI agents and automation are complementary, covering different workflow roles (e.g., diagnosis vs. monitoring) rather than duplicating each other. Third, only about one in five incidents contains a GFM; influence varies dramatically by functional role but far less by actor type: what a message says matters more than who posted it. We further show that GFMs can be surfaced automatically: σ-rank, an LLM ranker whose prompt encodes domain knowledge from the empirical study, places the GFM in the top 3 of ∼20 machine messages 89% of the time, reducing the engineer’s noise-reading burden by 7×. These results provide the first empirical evidence of how deployed AI output is actually consumed during live incidents and offer a practical mechanism to cut through information overload in time-critical incident response.

Article Search Article: ase26main-p860-p doi:10.1145/3832783.3837473

We Must Have Missed This Comment: Detecting and Repairing Stale Function References in Linux Kernel Comments
Kexin Sun, Yunbo Lyu, Xutong Ma, Hongyu Kuang, Ratnadira Widyasari, He Zhang, Xiaoxing Ma, Julia Lawall, and David Lo
(Nanjing University, China; Singapore Management University, Singapore; Inria, France)
As the Linux kernel evolves, code comments may become outdated, as the functions they reference can be refactored or removed independently without corresponding updates to the comments. Such stale function references can mislead maintainers and thus hinder code comprehension. Prior work on detecting code-comment inconsistency mainly focused on addressing semantic misalignment between Javadoc comments and their directly annotated functions, making them inapplicable to this type of externally induced staleness in the Linux kernel. Therefore, we propose ReCite, a three-stage approach to identify and repair such stale references: (1) detecting unresolved function-form symbols—symbols in comments that appear to reference functions but for which no matching function can be found in the current codebase, (2) tracing the evolution history of each unresolved symbol through the Git history, and (3) generating LLM-based repair suggestions grounded in the evolution history and current code context. On Linux kernel v6.18-rc1, ReCite detects 869 stale references with generated repair suggestions. A manual evaluation on 200 sampled repairs shows that 178 (89.0%) provide useful repair guidance, with 85 (42.5%) directly applicable. Of our 75 submitted patches, 50 have been accepted. We also empirically study all unresolved function-form symbols.

Article Search Article: ase26main-p908-p doi:10.1145/3832783.3837474

LARA: Static ReDoS Detection for Regular Expressions with Lookahead via Matching Automata
Mengxi Wang, Chengyao Peng, Weihao Su, Guiyi He, Rongchen Li, Hong Huang, and Haiming Chen
(Institute of Software at Chinese Academy of Sciences, China; University of Chinese Academy of Sciences, China)
Regular expressions (regexes) with lookahead assertions are widely used in modern software, yet their interaction with backtracking engines can cause Regular expression Denial of Service (ReDoS) vulnerabilities. Existing static methods rely on 𝜀-NFA ambiguity analysis, which cannot capture lookahead evaluation cost, while dynamic and hybrid methods incur high cost or incomplete coverage. We present LARA, a static ReDoS detector for regular expressions with lookahead (REwLA), which subsumes standard regexes. Our core contribution is the matching NFA, which transforms a REwLA into a standard 𝜀-NFA. The matching NFA is not language-equivalent to the original REwLA; however, we prove that its match degree equals the la-match degree of the original REwLA, so the same complexity upper bound is preserved, including lookahead overhead. This reduces lookahead-aware ReDoS detection to standard 𝜀-NFA ambiguity analysis, enabling existing detection techniques to be applied directly to the matching NFA. We evaluate LARA on 42,794 deduplicated real-world REwLA. Among 5,889 vulnerabilities jointly confirmed by multiple tools and validated on real backtracking engines, LARA identifies 5,871, outperforming all baselines. On 400 confirmed vulnerable regexes from 177 real-world ReDoS CVEs, LARA achieves 99.50% recall. LARA successfully exploits 4,468 vulnerabilities on real engines, with a median analysis time of 0.05 s and 7.67 MB memory.

Article Search Article: ase26main-p997-p doi:10.1145/3832783.3837475

ConfFuzz: Parameter-Aware Greybox Fuzzing for Configurable Cloud Systems
Shuai Wang, Hao Wang, Darko Marinov, Tianyin Xu, and Yongle Zhang
(University of Illinois at Urbana-Champaign, USA; University of California at Berkeley, USA; Purdue University, USA)
Cloud systems are highly configurable, with hundreds to thousands of configuration parameters. Configuration-related bugs are major sources of cloud failures, yet most tests only run under default configurations, leaving systems vulnerable to untested configurations. This paper applies coverage-based greybox fuzzing to configuration testing of cloud systems. We use developer-written unit tests as fuzz drivers: they avoid the cluster-initialization overhead of system tests and thus provide the high test execution speed that greybox fuzzing demands. Since cloud systems expose hundreds to thousands of configuration parameters but any unit test typically exercises only a small subset of them (15.14% on average), we design parameter-aware mutation and fuzzing energy allocation strategies: (1) we restrict mutation to only the configuration parameters exercised by each seed, and (2) we incorporate both code coverage and configuration parameter coverage into fuzzing energy allocation, giving more effort to seeds that explore more code and parameters. We implement these ideas in ConfFuzz and evaluate it on nine widely used cloud systems. Compared with a greybox fuzzing baseline without parameter-aware mutation and energy allocation, ConfFuzz covers roughly 3.2 times more branches while exploring substantially more configuration parameters. In total, ConfFuzz and its variants have detected 125 bugs, of which 14 have been fixed and 12 more have been confirmed.

Article Search Article: ase26main-p1001-p doi:10.1145/3832783.3837476

Synthesizing File-Level Data for Unit Test Generation with Chain-of-Thoughts via Self-Debugging
Ziyue Hua, Tianyu Chen, Yeyun Gong, Shuai Lu, Peng Cheng, Qinglin Zhu, Yibo He, Yingjie Fu, Wenpin Jiao, Wei Yang, and Tao Xie
(Peking University, China; Microsoft Research, China; King's College London, UK; Fudan University, China)
Automatic unit test (UT) generation is essential for software quality assurance, but existing approaches—including symbolic execution, search-based testing, and recent LLM-based generators—struggle to produce human-quality tests with correct, meaningful assertions and reliable chain-of-thought (CoT) explanations. We identify a gap in UT training data: repository-mined tests lack developer CoTs, while LLM-distilled CoTs are often incorrect or incomplete. To address this issue, we propose a novel data-distillation approach that uses self-debugging to produce high-quality UT training examples paired with faithful CoTs. Our approach combines (1) guided test repair, a heuristic loop (error-, failure-, coverage-, and mutation-focused steps) that prompts the same model to diagnose and iteratively fix generated tests, and (2) CoT compression, which compacts original and debugging CoTs into concise explanations that directly justify correct tests. We apply this pipeline to a large corpus of open-source projects to construct a dataset of 74,518 high-quality <focal method, test, CoT> examples, and then use it for supervised fine-tuning of a base model. An empirical evaluation shows that the fine-tuned model achieves high UT generation effectiveness: it attains a pass rate of 36.17% on test assertions, a branch coverage of 43.90%, and a mutation score of 88.66%, substantially higher than those of state-of-the-art commercial models such as o4-mini.

Article Search Article: ase26main-p1002-p doi:10.1145/3832783.3837477

FAiLGen: Calibrating LLM for Failing Test Generation with Monte Carlo Tree Search
Tsz-On Li, Xiaochuan Yan, Dehui Yi, Yuhang Jiang, Kexin Li, Congying Xu, Ying Wang, and Shing-Chi Cheung
(Hong Kong University of Science and Technology, Hong Kong; Northeastern University, China)
Failing test generation is an important yet challenging task in software engineering. It involves searching a vast space to identify fault-triggering test inputs and the oracles that assert these faulty executions. Several approaches have been proposed for generating failing tests. Specifically, given a program-under-test (PUT) and its requirements, these approaches leverage large language models (LLMs) to generate alternative implementations of the PUT that comply with the requirements, followed by differential testing between the PUT and the alternative implementations. Failing tests can be constructed by adopting the inconsistency-revealing inputs as the test inputs and treating the outputs returned by the alternative implementations as expected outputs. As alternative implementations may also contain faults and thus return incorrect outputs, current methods rely on either output consistency across alternative implementations or pre-constructed test cases for validation. However, the former is susceptible to LLM training biases or stochasticity, while the latter can be impractical due to the scarcity or unavailability of pre-constructed test cases in real-world scenarios. To address existing works’ limitations, we employ LLM as an alternative oracle for output validation, and calibrate its reasoning steps to reduce its susceptibility to LLM training biases or stochasticity. One critical challenge is that the calibration process inherently requires correct input-output pairs for the output validation task as ground-truth for reference when validating/calibrating LLMs’ reasoning, yet correct input-outputs (i.e., passing tests) are often scarce or unavailable in real-world scenarios. To address this challenge, our insight is that outputs consistently returned by both the PUT and all alternative implementations have a high probability of being the correct outputs of the PUT, as the PUT and alternative implementations are generated by independent agents (i.e., human developers and LLMs). Based on this insight, we propose FAiLGen, a novel approach that calibrates LLM’s reasoning for failing tests generation via a representative reasoning calibration paradigm, Monte Carlo Tree Search. We evaluate FAiLGen on 100 subjects from CodeForces and 100 subjects from SWE-Bench-Live. Our evaluation results show that FAiLGen achieves a recall of 58.1%, which is 62.4% higher than that of the best baseline (35.8%). Meanwhile, FAiLGen attains a precision of 75.2%, favorably compared to the best baseline (61.1%).

Article Search Artifacts Available Article: ase26main-p1022-p doi:10.1145/3832783.3837478

Feature-Focused Test Generation
José Antonio Zamudio Amaya, Gaetano Sapia, Alexi Turcotte, David Benavides, Marcel Böhme, and Andreas Zeller
(CISPA Helmholtz Center for Information Security, Germany; MPI-SP, Germany; Universidad de Sevilla, Spain)
Specification-based test generators use input specifications to produce valid tests for complex systems, covering the entire input space of the system under test. This broad generation capability, however, can become a burden when one wants to focus test generation on a particular subset of the input space, say, a feature whose implementation has recently changed or is particularly critical.
In this paper, we propose Feature-Focused Test Generation (FFTG), an approach that treats specification-based test generation as a configuration problem. From the input specification, FFTG first extracts a feature model that represents alternate and optional expansions as possible feature values. After the tester selects which features to focus upon, FFTG specializes the specification accordingly, generating inputs in which only the focused features vary.
We implement FFTG in a prototype named Compás. In our evaluation, we find that Compás-generated slices (1) effectively focus test generation on specific features, preserving validity; (2) exercise unique code locations associated with the targeted features and thus enable traceability of features; and (3) enable test suite minimization, with a small set of slices retaining the aggregate coverage of the full suite while requiring fewer tests.
To the best of our knowledge, this is the first work to apply product line engineering techniques to specification-based testing.

Article Search Article: ase26main-p1023-p doi:10.1145/3832783.3837479

SecChain: Operationalizing Security Guidelines for Multi-agent Smart Contract Generation
Dianxiang Sun, Liming Nie, and Yang Liu
(Nanyang Technological University, Singapore; Shenzhen Technology University, China)
Smart contracts generated by large language models (LLMs) can compile yet still violate basic security obligations, and existing generation frameworks largely treat security as post-hoc verification rather than as a constraint that persists throughout generation and repair. We present SecChain, a multi-agent smart-contract generation framework built around a persistent SecuritySpec artifact that makes requirement-level security obligations explicit and preserves them across planning, coding, testing, auditing, and fixing. Five LLM-powered agents collaborate with a fully deterministic Coordinator through stage-specific procedures, deterministic pre-test verification, and nested repair loops, while an Auditor provides complementary semantic review grounded in a three-layer security knowledge base. We evaluate SecChain on 1,000 contracts from the finite-state-machine-guided smart-contract generation (FSM-SCG) dataset against vanilla GPT-4o, GPT-5.4, Claude Sonnet 4.6, and the FSM-SCG framework. SecChain achieves an 89.0% final compilation rate and a Zero Risk Contract Proportion (ZRCP) of 60.3%, compared with 12.7% for vanilla GPT-4o and 23.3% for FSM-SCG. With a GPT-4o backbone, SecChain still outperforms the tested vanilla current-generation models (GPT-5.4, Claude Sonnet 4.6) on the primary security metrics.

Article Search Artifacts Available Article: ase26main-p1030-p doi:10.1145/3832783.3837480

SafeDep: Detecting State Mutation Unsoundness in Rust via Dependency Slicing and LLM Reasoning
Zihao Rao, Yilin Chen, Chengjun Chen, and Hui Xu
(Fudan University, China; Ant Group, China)
Rust's type system protects safe code, but many low-level abstractions depend on internal unsafe operations whose correctness rests on memory and logical invariants. In struct-based abstractions, a sibling method may legally modify internal fields while invalidating a precondition required by another method's unsafe operation. We call this defect an unsound state mutation. Traditional pattern-based static analyzers support inter-procedural tracking but may lack the capability to model cross-method state interference. Furthermore, they cannot interpret natural-language safety contracts, making these deep defects difficult to detect. LLMs can reason about these contracts, but whole-crate prompting is expensive and often prone to hallucinated paths. To bridge this gap, we present SafeDep, a vulnerability discovery framework that combines compiler-driven static analysis with LLM-based semantic reasoning. SafeDep analyzes Rust MIR to build a field-sensitive read/write dependency graph and slices the graph to retain methods that may affect safety-critical state. An LLM then reconstructs cross-method state transitions from this compact context and generates PoC candidates. A bounded compiler/Miri feedback loop refines the candidates and their exploitability judgments before manual audit. On our evaluation dataset, SafeDep outperforms static and LLM baselines while reducing prompt context. A scan of the top 4,000 crates.io projects found 24 previously unreported soundness vulnerabilities, 18 of which had been confirmed by maintainers or accepted as RustSec advisories at the time of writing.

Article Search Artifacts Available Article: ase26main-p1035-p doi:10.1145/3832783.3837481

A Selective Undo/Redo Algorithm for Dependency-Constrained Operation Histories Based on Change Propagation
Xiaoxiong Fan, Hou Tam, Hanxi Zhu, and Song-Hai Zhang
(Tsinghua University, China)
Selective undo and redo let users disable or re-enable a specific historical operation while attempting to preserve unrelated later work. This is particularly valuable in interaction-heavy and collaborative editing environments, where mistakes may be discovered only after substantial later work has accumulated and where discarded edits are costly to reconstruct manually. Achieving this is significantly harder when operations are dependency-constrained: whether they remain valid after a toggle may depend on complex semantic conditions, and naively replaying long history suffixes after every toggle is too expensive for interactive use. To this end, we present a dependency-sensitive change-propagation algorithm that preserves a replay-style correctness target without full replay after every toggle. Our approach decomposes editor state into fine-grained slots with reducer-backed semantics, evaluates operation legality against prefix state, and propagates the effect of a toggle only through the affected neighborhood of later operations and state changes. We implement the mechanism in the scene editor of an in-house game IDE built on a heavily modified version of the Godot Engine. Across differential testing against two exact replay baselines and a validity-aware fuzzing campaign comprising 45 runs and 450,000 recorded actions, our change-propagation pipeline matched the reference semantics on all evaluated workloads. It substantially reduced toggle latency and semantic work relative to both baselines, including a reduction of about 194× in prerequisite evaluations. These results show that replay-style selective undo/redo can be made practical for operations with hard dependency constraints.

Article Search Article: ase26main-p1098-p doi:10.1145/3832783.3837482

WiDepFuzz: Finding Wi-Fi Protocol Bugs in Mobile Hotspot via Field Dependency Model
Yuanliang Chen, Fuchen Ma, Taotao Gu, and Yu Jiang
(Tsinghua University, China)
Wi-Fi is a widely used wireless protocol, and mobile hotspot functionality has become essential for on-the-go connectivity. Bugs in the hotspot protocol stack can cause crashes, performance degradation, and semantic inconsistencies. Fuzzing is a promising technique for detecting such vulnerabilities. However, existing fuzzers typically test by modeling protocol inputs but ignore some critical dependencies among Wi-Fi fields, leading to inefficient testing.
In this work, we present WiDepFuzz, a fuzzing framework designed to detect bugs in Mobile Hotspot implementations by modeling field dependencies in the Wi-Fi protocol. First, WiDepFuzz automatically identifies the semantic dependencies among Wi-Fi fields based on the IEEE 802.11 standard and constructs a Field Dependency Model (FDModel) to characterize various direct and indirect dependencies. Second, leveraging the FDModel, WiDepFuzz utilizes dependency-combination guided fuzzing to constantly generate Wi-Fi test frames that efficiently explore deep paths and trigger hidden bugs. Finally, we propose liveness and dependency detectors that enable real-time monitoring to identify abnormal states and uncover potential bugs in mobile hotspots. We implemented WiDepFuzz and evaluated it in six widely used mobile hotspots, covering major smartphone manufacturers, including Samsung, Xiaomi, OPPO, Honor, Huawei, and Nokia. Compared to SOTA Wi-Fi fuzzers like Peach, OwFuzz, wpaspy, and Greyhound, WiDepFuzz detects 15 more bugs and improves bug detection speed by 37% on average. In total, it uncovered 20 new bugs.

Article Search Article: ase26main-p1165-p doi:10.1145/3832783.3837483

CTForge: Automatically Generating Test Suites for Software Configuration
Yuanliang Zhang, Zhizheng Zheng, Shanshan Li, Zhouyang Jia, Chaopeng Luo, Liqian Chen, Zhenbang Chen, Ji Wang, and Xiangke Liao
(National University of Defense Technology, China)
Software configuration plays a critical role in modern software systems, yet it also frequently leads to severe failures. Existing approaches, whether static analysis or LLM-based validation, struggle to capture implicit semantic constraints and validate configuration-code interactions. Leveraging existing unit tests is also insufficient due to their lack of configuration-specific focus. Thus, we present CTForge, an LLM-powered framework that automatically generates configuration-specific test suites. Its core innovation is a three-stage configuration-aware optimization pipeline that moves beyond vanilla test generation, specifically using parameter semantics to improve LLM outputs. This optimization pipeline iteratively aligns tests through: (1)enhance configuration-related code coverage to ensure relevance, (2) assertion validation across valid parameter values to eliminate false assertions, and (3) configuration-sensitivity verification to select tests that semantically exercise the intended configuration behavior. Evaluated on five large-scale systems, CTForge detects 63 out of 79 real-world configuration issues and achieves an F1-score of 0.88 in misconfiguration detection. CTForge also uncovers six previously unknown defects. Our work demonstrates that structured configuration-aware refinement is essential for LLM to produce effective test suites.

Article Search Article: ase26main-p1197-p doi:10.1145/3832783.3837484

Panorama: Unveiling Latent Dependencies for Microservice Autoscaling via Meta-learning
Zhuangbin Chen, Hongjiang Feng, Yang Liu, Juzheng Zheng, Xiaoyu He, and Zibin Zheng
(Sun Yat-sen University, China)
Existing autoscaling approaches primarily leverage direct service calls to model the propagation effects of scaling decisions across inter-service dependencies. However, the intricate and implicit service interactions (i.e., latent dependencies) often go beyond such direct invocations, rendering suboptimal scaling actions and poor adaptability. To address these limitations, we propose Panorama, a novel autoscaling framework that explicitly unveils and exploits indirect service interactions. In particular, we formalize two fundamental categories of latent dependencies, i.e., resource interference dependencies (RIDs), which emerge from shared infrastructure resources, and execution blocking dependencies (EBDs), which stem from temporal constraints in the execution logic of business workflows. Panorama employs a hybrid attention mechanism to model these multifaceted service dependencies alongside temporal dynamics. We also design a meta-learning paradigm to enable rapid adaptation to evolving workload patterns and dependency structures without extensive retraining. Evaluation across four representative microservice benchmarks shows that Panorama consistently outperforms state-of-the-art baselines, achieving up to 68% reduction in resource consumption while improving response time by 42%. This highlights the critical importance of explicitly modeling latent dependencies for effective microservice resource management.

Article Search Article: ase26main-p1217-p doi:10.1145/3832783.3837485

ReCodeAgent: A Multi-agent Workflow for Language-Agnostic Translation and Validation of Large-Scale Repositories
Ali Reza Ibrahimzada, Brandon Paulsen, Daniel Kroening, and Reyhaneh Jabbarvand
(University of Illinois at Urbana-Champaign, USA; Amazon, USA)
Most repository-level code translation and validation techniques have been evaluated on a single source-target programming language (PL) pair, owing to the complex engineering effort required to adapt new PL pairs. Programming agents can enable PL-agnosticism in repository-level code translation and validation: they can synthesize code across many PLs and autonomously use existing tools specific to each PL’s analysis. However, state-of-the-art has yet to offer a fully autonomous agentic approach for repository-level code translation and validation of large-scale programs. This paper proposes ReCodeAgent, an autonomous multi-agent approach for language-agnostic repository-level code translation and validation. Users only need to provide the project in the source PL and specify the target PL for ReCodeAgent to automatically translate and validate the entire repository. ReCodeAgent is the first technique to achieve high translation success rates across many PLs. We compare the effectiveness of ReCodeAgent with four alternative neuro-symbolic and agentic approaches to translate 118 real-world projects, with 1,975 LoC and 43 translation units for each project, on average. The projects cover 6 PLs and 4 PL pairs. Our results demonstrate that ReCodeAgent consistently outperforms prior techniques on translation correctness, improving test pass rate by 60.8% on ground-truth tests, with an average cost of $15.3. We also perform process-centric analysis of ReCodeAgent trajectories to confirm its procedural efficiency. Finally, we investigate how the design choices (a multi-agent vs. single-agent architecture) influence ReCodeAgent performance: on average, the test pass rate drops by 40.4%, and trajectories become 28% longer and persistently inefficient.

Article Search Article: ase26main-p1257-p doi:10.1145/3832783.3837486

Aligning User Requirement with Large Language Model in Code Generation
Jia Li, Ruiqi Bai, Yangkang Luo, Yiran Zhang, Wentao Yang, Zeyu Sun, Tiankuo Zhao, Dongming Jin, Lei Li, and Zhi Jin
(Wuhan University, China; Nanyang Technological University, Singapore; Institute of Software at Chinese Academy of Sciences, China; Peking University, China; University of Hong Kong, Hong Kong)
Code generation refers to automatically producing executable programs from user requirements. Recently, researchers have explored approaches to enhance the correctness of generated code with advanced large language models. Although achieving improvements, existing approaches focus on designing reasoning strategies or post-refinement methods to enhance code generation performance. Despite their differences, all these methods share a common assumption: the LLM can correctly understand the given requirement. However, this assumption does not always hold. To fill this gap, we propose REA-Coder, a requirement alignment approach to enhance the code generation performance of LLMs. REA-Coder involves first identifying the requirement content that does not align with LLMs and aligning the requirements. Then, based on the aligned requirements, LLMs generate code and further verify whether the generated code aligns with the requirements, iterating this process of requirement alignment and code generation until generating correct code or achieving the maximum number of iterations. Experimental results show that REA-Coder outperforms all advanced baselines on four LLMs across five programming benchmarks. Concretely, REA-Coder achieves average improvements of 7.93%, 30.25%, 26.75%, 8.59%, and 8.15% on the five benchmarks, demonstrating the effectiveness of requirement alignment for improving the code generation performance of LLMs.

Article Search Article: ase26main-p1298-p doi:10.1145/3832783.3837487

QDoctor: Probe-Based Testing for Quantum Programs
Qichen Wang, Shangzhou Xia, Xiaoyu Guo, and Jianjun Zhao
(Kyushu University, Japan)
Testing quantum programs in a black-box setting is difficult because bug-relevant behavioral differences may be missed at two levels: the chosen inputs may fail to trigger faulty behavior, and even when faults are triggered, the chosen measurements may fail to make the resulting differences observable. Existing methods have mainly strengthened either input generation or measurement design in isolation, leaving their coupling insufficiently addressed. In this paper, we present QDoctor, a probe-based black-box testing framework that wraps the program under test with a variety of property-based probes, thereby effectively extracting program information and exploring its behavior. A probe integrates two components: test-case generation and the specification of the measurement basis. To reduce false positive rate, QDoctor also supports semantic-aware partial equivalence testing through constrained probe-oracle design. Experimental results on benchmark suites show that QDoctor achieves the best balanced performance among representative black-box baselines, reaching 99.89% recall, 98.23% precision, 99.06% F1, and 0.9988 AUC. Ablation shows that jointly strengthening inputs and measurements raises recall from 85.280% to 99.728% with zero false positives, while semantic probes drive the false-positive rate to zero on SWAP-induced equivalent mutants.

Article Search Article: ase26main-p1302-p doi:10.1145/3832783.3837488

Doc2Feat-Bench: Evaluating Documentation-Driven Feature Addition
Zhonghao Jiang, Le Deng, Jialun Cao, Michael Pradel, and Zhongxin Liu
(Zhejiang University, China; Hong Kong University of Science and Technology, Hong Kong; CISPA Helmholtz Center for Information Security, Germany)
Documentation changes in mature software projects often describe newly introduced or modified behavior. This makes them a natural basis for documentation-driven feature addition, where software engineering agents implement features from public-facing documentation updates. However, existing benchmarks for agentic software development primarily rely on issue reports written for maintainer coordination and often contain implementation-level details. Motivated by this observation, this work introduces Doc2Feat-bench, a benchmark for documentation-driven feature addition. Doc2Feat-bench consists of 634 tasks across 10 mature open-source software projects, involving about 114k code changes in total. Each task pairs a documentation change with the corresponding implementation and developer-written tests, enabling evaluation of whether a software engineering agent can implement the feature described by the documentation diff. The benchmark is constructed through a five-phase pipeline, starting from release notes, and offers broad coverage across libraries, developer tools, and frameworks while preserving real-world development settings. To facilitate lightweight and reliable evaluation under limited resources, we further curate a human-validated subset named Doc2Feat-bench Verified. It contains 114 high-quality instances whose task clarity and evaluation validity are manually verified. We use Doc2Feat-bench to assess a range of state-of-the-art software engineering agents with two widely used scaffolds. Experimental results show that despite significant token consumption, the best task success rate remains as low as 37.72% (OpenHands with Qwen3-Coder-480B). Our analysis reveals that software engineering agents face key challenges in performing cross-file edits, understanding existing code modules, and correctly interpreting documentation semantics.

Article Search Article: ase26main-p1306-p doi:10.1145/3832783.3837489

RegoMender: A Multi-agent Framework for Automated Rego Policy Repair
Ruijie Zhang, Huaxiao Yin, Li Hu, Fanqi Kong, Haozhe Liang, Shaoyin Cheng, Kun Zhang, and Bibo Tu
(Institute of Information Engineering at Chinese Academy of Sciences, China; University of Science and Technology of China, China; Peking University, China; University of Chinese Academy of Sciences, China)
Policy-as-Code (PaC) has emerged as a foundational paradigm in cloud-native systems. However, repairing Rego policies remains challenging due to their strict syntax, declarative semantics, and specific verification requirements. Existing automated program repair (APR) methods are mainly designed for mainstream languages and struggle to accommodate the unique policy logic and repair workflows of Rego. Thus, Rego policy repair remains a manual process driven by trial and error. To address this, we introduce RegoBench, the first automated Rego policy benchmark comprising 1,159 real-world defects, and propose RegoMender, a feedback-guided multi-agent framework. Collaborating via a shared blackboard and an atomic toolkit, RegoMender dynamically executes patching, rewriting, or rolling back, to resolve complex repair scenarios. Evaluated on RegoBench, RegoMender achieves syntactic and test-based semantic success rates of 82.9% and 77.9%, respectively, outperforming the strongest baseline by 13.4%. Unlike other approaches whose performance drops in complex scenarios, RegoMender maintains effectiveness across single-step and chained repair tasks. Furthermore, it generalizes strongly across HCL and SQL and achieves an 86.4% repair rate on 132 real-world CVE vulnerabilities, proving its robustness and practical utility for security.

Article Search Article: ase26main-p1325-p doi:10.1145/3832783.3837490

Defensive Capability Analysis for JavaScript Libraries
Wenyuan Xu and Anders Møller
(Aarhus University, Denmark)
Capability analysis is a key building block for JavaScript supply-chain security tasks such as malware detection, security auditing, and runtime policy construction. However, existing approaches do not provide guarantees in the presence of JavaScript’s dynamic features.
We present a defensive capability analysis for JavaScript libraries that soundly reports every exercised capability for code executed under a lightweight protected runtime. The analysis tracks only objects that are relevant to capabilities. It reports capability usage either at the program locations where such objects are used, or conservatively when relevant objects escape through operations that the analysis does not model precisely. To address a small number of dynamic language features that would otherwise invalidate this analysis, we complement the static analysis with a lightweight runtime enforcement mechanism that blocks those patterns.
We implement the approach and evaluate it on large datasets of benign and malicious npm packages. The results show that the approach scales to real-world packages, pinpoints capability-usage locations with high accuracy, and remains compatible with 98.2% of benign libraries when using the protected runtime. Moreover, it detects capability usage in 99.8% of the malicious packages in our dataset, while the remaining cases are blocked by the runtime protection. Our study also shows that at least 72.9% of the npm packages use no security-sensitive capabilities at all, suggesting that capability analysis can substantially reduce auditing effort in practice.

Article Search Artifacts Available Article: ase26main-p1332-p doi:10.1145/3832783.3837491

RAVEN: Measuring Detection Instability in Anti-virus Engines under Semantics-Preserving Perturbations
Younghoon Ban, Beomjin Jin, Doowon Kim, Hyoungshick Kim, and Haehyun Cho
(Soongsil University, Republic of Korea; Ajou University, Republic of Korea; University of Tennessee, Knoxville, USA; Sungkyunkwan University, Republic of Korea)
Real-world anti-virus (AV) engines continue to rely heavily on signature-based detection, whereas academic adversarial research has largely focused on ML-based models, leaving an evaluation gap. To better understand signature-based detection behavior, we analyze ClamAV’s open-source signature database as a white-box reference point. Our analysis shows that raw byte-level signatures account for 94.54

Article Search Article: ase26main-p1356-p doi:10.1145/3832783.3837492

CiL-Agent: Compiler-in-the-Loop LLM Agent for Memory-Safe C Program Repair
Yang Shen, Yating Zhang, Jiaqiang Yao, Minghao Liu, and Wei Dong
(National University of Defense Technology, China)
Automated Program Repair (APR) has advanced rapidly on managed languages such as Java and Python, yet C—the bedrock of systems software—remains far harder to repair safely. The difficulty is specific to C: as an unmanaged language with manual memory management, a patch can compile and pass every functional test while silently introducing a memory-safety defect (a leak, use-after-free, or double free) that managed-language tests would never expose, making such “fixes” potentially worse than no fix. Existing LLM-based repair offers little defense, as it learns from impoverished feedback—no execution at all (APPATCH), binary pass/fail signals (ChatRepair), or loosely-structured, unscored sanitizer text (VRpilot). We present CiL-Agent (Compiler-in-the-Loop Agent), which tightly integrates GCC, the Clang Static Analyzer, and Valgrind into the LLM generation loop, parsing their diagnostics into structured, three-dimensional feedback: compilation reachability, static provability, and dynamic memory safety. A two-phase strategy first samples and composite-scores candidate patches (Best-of-N), then refines the best under precise tool feedback. On 1,000 real-world C vulnerabilities spanning 11 CWE categories, CiL-Agent attains 88.2% Pass@1 with 87.3% memory-safe patches, outperforming five repair baselines (by 8.3–58.4pp, all p<0.001, McNemar’s test) and five modern coding agents by a wide margin. A full 1,000-task ablation shows toolchain integration and iterative refinement are independently essential (−44.3 and −35.2 percentage points), with toolchain integration the single most critical component, and a post-cutoff benchmark confirms the gains are not due to training-data memorization.

Article Search Article: ase26main-p1378-p doi:10.1145/3832783.3837493

Safe Multi-site Binary Instrumentation of Running x86-64 Programs
Jihun Baek, Sanghoon Han, and Hyungon Moon
(UNIST, Republic of Korea)
Applying binary instrumentation to a running x86-64 process is difficult because variable-length instructions and weak alignment guarantees prevent safe in-place code modification under concurrent execution. Existing run-time approaches either trap concurrent threads on every overwrite or require global synchronization, limiting scalability. This paper presents DIBE, a run-time binary editing framework that safely instruments running x86-64 programs without trapping in the common case. An admin process resolves all instrumentation decisions, including multi-site patching tactics and trampoline placement, and exports them as metadata. A lightweight runtime inside the target process replays these edits, safely activating E9Patch's multi-site rewriting tactics on the live process by applying their dependent overwrites in an order that keeps every intermediate program state correct under concurrent execution, a guarantee that single-site punning and offline rewriting do not provide. It applies 77-90% of overwrites using hardware atomic primitives invisible to concurrent threads, with a trap-based fallback only for the remaining cross-boundary cases. We evaluate DIBE on 28 SPEC CPU2006 benchmarks and a multi-process Nginx server, and demonstrate it at browser scale on a running Chrome renderer and as a live memory-safety checker. DIBE matches static E9Patch on both instrumentation coverage (near-complete) and execution time overhead (within 0.5% at the geometric mean), keeps per-instruction overwrite latency sub-microsecond, and records zero trap hits across all Nginx configurations under sustained load.

Article Search Article: ase26main-p1407-p doi:10.1145/3832783.3837494

Mobile Bug Reproduction via Global State Reprioritization and LLM-Guided Trajectory Exploration
Dingbang Wang, Sidong Feng, William G. J. Halfond, and Tingting Yu
(University of Connecticut, USA; Monash University, Australia; University of Southern California, USA)
Mobile bug reproduction is essential to effective debugging in software engineering. Recent methods leveraging Large Language Models (LLMs) have shown significant performance gains over traditional techniques in automating this task. However, existing LLM-based methods remain constrained by two major limitations. First, they lack robustness to unreliable or inconsistent LLM outputs due to overreliance on a single LLM-suggested action without principled planning. Second, substantial token overhead arises from frequent LLM queries and the limited utilization of each query’s output capacity. To address these limitations, we propose ReproPilot, a lightweight framework that combines global planning with LLM-based semantic reasoning to reproduce mobile bugs. Specifically, the LLM provides diverse, reusable long-horizon guidance beyond step-wise action suggestions. This guidance is integrated with heuristic signals by a global planning mechanism to reason about reproduction progress and globally determine where to explore (i.e., which reproduction states) and what trajectories to explore, rather than merely executing actions on the current UI page at each iteration. As a result, ReproPilot improves the consistency and cost-efficiency of bug reproduction while maintaining competitive effectiveness. Evaluation on 74 real-world Android crash reports shows that our approach delivers the best reliability–cost trade-off among the evaluated methods, attaining the highest consistency (80.14%) with competitive effectiveness (86.49%), reducing reproduction time by 2.32%–14.07%, and lowering LLM token consumption by 65.39%–83.96% compared with state-of-the-art LLM-based baselines.

Article Search Article: ase26main-p1450-p doi:10.1145/3832783.3837495

IncSFS: Incremental Full-Sparse Flow-Sensitive Pointer Analysis for C/C++
Kunlin Liu, Zhenbang Chen, Piyi Zu, Yide Du, and Ji Wang
(National University of Defense Technology, China)
Pointer analysis is the fundamental technique for compiler optimization and program analysis. Flow sensitive pointer analysis provides high precision but hard to scale to large-size projects. Tailored for rapid iteration scenarios where software evolves continuously, we introduce IncSFS, the first incremental full-sparse flow-sensitive pointer analysis algorithm for C/C++ programs. The algorithm contains two main steps. It first transforms the value-flow graph of a program into a constraint graph, on which a strongly-connected component detection is performed to ensure precision. It then propagates increases and decreases in points-to set in an interleaving manner to support both code deletion and insertion within a single analysis pass. IncSFS is guaranteed to terminate and compute the least fixed point when point-to relation during the analysis is object-acyclic. Experimental results on six large-scale real-world projects show that IncSFS is both precise and efficient, achieving average speedups of 9.60 x over full flow-sensitive pointer analysis and of 5.84 x over the traditional reset-recompute incremental approach. Furthermore, IncSFS improves efficiency by 15.8% compared with state-of-the-art incremental pointer analysis algorithms that also propagate changes in points-to sets.

Article Search Artifacts Available Article: ase26main-p1456-p doi:10.1145/3832783.3837496

ExcavatorFuzz: LLM-Guided Path-Difficulty-Driven Directed Greybox Fuzzing
Kaiyu Xie, Siyuan Li, Jiaqian Peng, Yun Yang, Yaowen Zheng, Kai Cheng, Jiayu Zhang, Shizhao Tian, Yubo Li, and Hongsong Zhu
(Institute of Information Engineering at Chinese Academy of Sciences, Beijing, China; Shandong University, China)
Directed greybox fuzzing (DGF) is effective at steering mutations toward vulnerable code regions, but its efficiency is often limited by two bottlenecks: misprioritized target-reaching paths and hard-to-satisfy blocking predicates. We present ExcavatorFuzz, an LLM-assisted DGF framework that addresses both challenges in a unified workflow. ExcavatorFuzz first performs CLI-aware feasibility filtering to prune configuration-infeasible paths, and then ranks the remaining candidate paths using a lightweight difficulty metric that approximates their relative tractability based on explicit predicate complexity and implicit data-dependency complexity. During fuzzing, ExcavatorFuzz monitors line-level progress to detect blocked paths and invokes the LLM to synthesize or refine seeds for unsatisfied predicates using structured feedback. We evaluate ExcavatorFuzz on 30 real-world C/C++ benchmarks. ExcavatorFuzz triggers 29 out of 30 known vulnerabilities within 24 hours and achieves an average 4.34x speedup in time-to-exposure over seven state-of-the-art fuzzers. It also discovered three previously unknown vulnerabilities in open-source projects, two of which have been assigned CVE IDs.

Article Search Article: ase26main-p1484-p doi:10.1145/3832783.3837497

GANADI: Uncovering C/C++ OSS Reuse Genealogies via Pivotal Function-Based Clustering to Enhance Supply Chain Security
Dongyeon Kim, Seunghoon Woo, and Heejo Lee
(Korea University, Republic of Korea)
We present GANADI, a systematic approach for identifying C/C++ OSS reuse genealogies to enhance software supply chain security. Understanding OSS reuse genealogy is crucial for improving SBOM completeness and prioritizing security remediation across supply chains. Although existing approaches can identify reused components and vulnerabilities within a project, they fail to trace OSS reuse paths through intermediate projects, limiting their effectiveness in securing supply chain ecosystems. To address this limitation, GANADI constructs reuse genealogies by clustering downstream projects based on shared characteristics of origin-derived code (called pivotal functions), and then inferring reuse direction among the projects within each cluster. When applied to 20 widely reused OSS projects with over 1,500 propagation paths, GANADI achieved 84.85% precision and 95.76% recall in identifying reuse genealogies, outperforming existing approaches that achieved at most 23.21% recall. Leveraging OSS reuse genealogy for vulnerability detection, we identified 48 unpatched vulnerabilities in real-world popular C/C++ projects. Among them, 23 were patched following our responsible disclosure (including one CVE ID assigned), demonstrating the practical impact of genealogy-based vulnerability management.

Article Search Artifacts Available Article: ase26main-p1485-p doi:10.1145/3832783.3837498

OrthCoder: Balancing Security and Utility in Secure Code Generation via Orthogonal Adaptation
Menghan Tian, Youliang Yuan, Zhiqing Zhong, and Pinjia He
(Chinese University of Hong Kong, Shenzhen, China)
Large Language Models (LLMs) have significantly advanced automated code generation, yet they frequently produce code with critical security vulnerabilities. While recent alignment techniques (e.g., supervised fine-tuning and preference optimization) have attempted to mitigate this, existing methods often suffer from the notorious safety tax—a severe degradation in general functional utility, particularly on complex programming tasks. This trade-off can arise from parameter interference, where security-driven updates interact with the model’s pre-trained functional representations. Alternative approaches that avoid modifying model parameters (e.g., inference-time methods) can alleviate this issue but often introduce additional inference overhead or instability.
In this work, we focus on snippet-level secure code generation, where a model generates a self-contained function or short code routine from a natural-language prompt. Within this setting, we propose OrthCoder, a novel secure code generation framework based on subspace decoupling. OrthCoder employs a Dual-LoRA architecture, dedicating separate adapters to functional utility and security alignment. By introducing a geometric orthogonality constraint (L_orth) during joint training, we encourage the utility and security adapters to learn less-overlapping parameter updates.
Extensive evaluations across four diverse foundational models (∼7B scale) on seven benchmarks covering snippet-level security and general code-generation utility demonstrate that OrthCoder achieves state-of-the-art performance on joint security-utility benchmarks such as CodeGuard+ and CWEval. More importantly, OrthCoder mitigates the severe safety tax observed in existing alignment baselines. Furthermore, analyses of the learned adapter parameters show substantially reduced overlap between utility and security updates, supporting the effectiveness of the proposed decoupling mechanism.

Article Search Artifacts Available Article: ase26main-p1510-p doi:10.1145/3832783.3837499

Zero-Knowledge VMs: Security and Usability in Practice
Jinan Jiang, Jinzhao Chu, Pengyu Xue, Xinghao Peng, Haoran Qin, and Xiapu Luo
(Hong Kong Polytechnic University, Hong Kong)
Zero-knowledge Virtual Machines (zkVMs) are emerging critical infrastructure for blockchain Layer 2 scaling solutions, enabling execution of arbitrary programs while generating succinct proofs of correct computation. Despite hundreds of millions of investment, the rapidly evolving zkVM ecosystem lacks comprehensive empirical analysis of its security and usability challenges. We present the first systematic study of zkVM vulnerability and usability issues, analyzing production zkVMs through three perspectives covering 314 security vulnerabilities (professional audits), 11,854 commits (developer practices), and 1,150 user issues (user experiences). We identify prevalent, severe, and non-traditional vulnerability categories and their root causes, reveal high-value attack surfaces that consistently emerge across zkVMs, and identify major usability issues that users struggle with. Comparing our findings against existing research reveals four gaps representing research and tool development opportunities for the testing and analysis of zkVMs.

Article Search Article: ase26main-p1549-p doi:10.1145/3832783.3837500

Formal Verification-Integrated Reinforcement Learning for Automated ORM Schema Synthesis
Md Rashedul Hasan and Hamid Bagheri
(University of Nebraska-Lincoln, USA)
Object-relational mapping (ORM) frameworks are widely used to bridge object-oriented models and relational databases, but conventional tools generate only single point solutions and often miss optimal trade-offs between performance and storage consumption. Exhaustive tradespace analysis can discover Pareto-optimal database designs, but it is computationally prohibitive for realistic systems. This paper introduces ReinforceORM, an automated and scalable framework that formulates ORM schema synthesis as a multi-objective optimization problem solved via reinforcement learning with formal verification. Our approach integrates multiple RL algorithms to efficiently search the vast space of mapping strategies, balancing insertion time, query performance, and storage consumption, while employing Alloy-based formal analysis to provide bounded verification of structural and semantic correctness in generated schemas. Evaluation across fifteen diverse, real-world software systems shows that ReinforceORM identifies over 80% of ground-truth Pareto-optimal ORM configurations—an 18-fold improvement over leading ORM frameworks such as Hibernate, Django ORM, SQLAlchemy, and Entity Framework. Compared to state-of-the-art specification-driven search, ReinforceORM reduces analysis runtime and memory usage by four and five orders of magnitude, respectively, while maintaining near-optimal solution quality.

Article Search Artifacts Available Article: ase26main-p1565-p doi:10.1145/3832783.3837501

Repository-Aware Metamorphic Relation Generation for Augmented Reality Applications using Large Language Models
Dibyendu Brinto Bose, Jiawei Qin, and Chris Brown
(Virginia Tech, USA)
Metamorphic Testing (MT) provides a promising approach for testing software without defined test oracles by specifying expected relations between inputs and outputs, instead of relying on exact outputs. For example, testing Augmented Reality (AR) applications is challenging due to dynamic interactions between virtual content, physical environments, and code, which make traditional test oracles difficult to define. However, formulating metamorphic relations (MRs) is time-consuming and burdensome. We introduce a context-aware pipeline that generates and refines MRs using repository-level context and reasoning orchestration, evaluated on a dataset of 142 mobile AR system repositories. Across three context configurations generating 14,916 candidate MRs, hierarchical context yielded the broadest coverage (7,004 MRs across 142 repositories and 5,167 class–method pairs) and lower redundancy. An agentic deliberation process then reconciled conflicting candidates—observed in 79.0% of cases—reducing duplication and selecting context-aware relations in 88.2% of outcomes. A manual oracle study shows refined relations (n = 141) are both logically valid and sufficiently concrete to be directly translated into test assertions, and a preliminary case study reveals converting generated MRs (n = 5) into executable tests can detect non-equivalent mutations in real-world code. Overall, our results show that combining repository-aware MR generation with reasoning-based refinement enables scalable construction of reliable, domain-relevant test oracles.

Article Search Article: ase26main-p1578-p doi:10.1145/3832783.3837502

Learning to Triage Vulnerability Reports from Program Analysis: An Empirical Study in Node.js
Ronghao Ni, Aidan Z.H. Yang, Min-Chien Hsu, Nuno Sabino, Limin Jia, Ruben Martins, Darion Cassel, and Kevin Cheang
(Carnegie Mellon University, USA; Athanor AI, USA; University of California at Berkeley, USA)
Program analysis tools often produce large volumes of candidate vulnerability reports that require costly manual review, creating a practical challenge: how can security analysts prioritize the reports most likely to be true vulnerabilities? This paper investigates whether machine learning can be applied to prioritizing vulnerabilities reported by program analysis tools. We focus on Node.js packages and collect a benchmark of 1,883 Node.js packages, each containing one reported ACE or ACI vulnerability. We evaluate a variety of machine learning approaches, including classical models, graph neural networks (GNNs), large language models (LLMs), and hybrid models that combine GNNs and LLMs, trained on data derived from program analysis tool outputs (NodeMedic-FINE in our case study). The top LLM achieves F₁=0.915, while the best provenance-graph-based method achieves F₁=0.904. On the reports that the upstream tool flags but cannot automatically confirm, at a target of recovering 90% of the exploitable reports, the leading model eliminates 75% of the benign reports from manual review. If the best model is tuned to operate at a precision level of 0.8 (i.e., allowing 20% false positives among all warnings), our approach can report 99.2% of exploitable taint flows while missing only 0.8%, demonstrating strong potential for real-world vulnerability triage.

Article Search Article: ase26main-p1614-p doi:10.1145/3832783.3837503

Code Transformation Rule Synthesis using LLMs: Potential and Limits
Axel Allain, Aymeric Blot, Djamel Eddine Khelladi, and Mathieu Acher
(INSA, France; University of Rennes, France; Inria, France; CNRS, France; IRISA, France)
Due to their black-box nature, LLMs suffer from limited explain- ability and a lack of determinism. Their usage cost can also rise, particularly with repetitive tasks on large codebases. Code transfor- mation rules mitigate those challenges. This paper fills this gap, with a novel empirical study targeting three domain-specific languages for transformation rules, namely Comby, GritQL, and Ast-Grep We evaluate three LLMs (GPT-5.4, GPT-oss-120B, and Llama3.1-8B) on six diverse datasets covering four software-evolution tasks: API misuse correction, program repair, API migration, and language ver- sion migration. Our results provide evidence that transformation rule synthesis moves beyond proof-of-concept with strong frontier models. GPT-5.4 achieves consistently high rule applicability rates and produces transformations closest to the ground truth across most benchmarks. Smaller and open-weight GPT-oss-120B and Llama3.1-8B models remain effective for simpler, localized changes but struggle with complex migration scenarios. We also observe non-negligible generalizability through the usage of meta-variables and through a high reuse score in the first quartile of many datasets. Finally, when compared to the anti-unification algorithm, LLMs outperform it in correctness, but underperform in rule applicabil- ity. Overall, our results show great potential for LLMs to generate sound, correct, generalizable, and reusable rules.

Article Search Artifacts Available Article: ase26main-p1631-p doi:10.1145/3832783.3837504

A Low-Cost Human-in-the-Loop Investigation of Toxicity on GitHub at Scale
Rahat Rizvi Rahman, Mia Mohammad Imran, and Kostadin Damevski
(Virginia Commonwealth University, USA; Missouri University of Science and Technology, USA)
Toxic interactions in open source discussions can alienate contributors and threaten project sustainability, yet prior empirical studies of GitHub toxicity have been limited in scale, raising questions about their generalizability. Scaling up is difficult because toxicity on GitHub is often implicit and context-dependent, making both fully manual annotation and LLM-based labeling unreliable.
We present a human-in-the-loop (HITL) annotation methodology that makes large-scale, domain-calibrated toxicity labeling practical. A single call to a small, local LLM produces both a toxicity prediction and a set of interpretable event category scores. A lightweight Random Forest validator then uses those scores to flag the small subset of conversations most likely to be mislabeled, directing human review only where it is needed. The validator outperforms confidence-based and multi-LLM baselines while adding low annotation cost.
We apply this pipeline to over 124,000 GitHub issue and pull request conversations. Using the resulting dataset, we evaluate key findings from prior small-scale research, confirming some and qualifying others, and present new insights into the prevalence, characteristics, and dynamics of toxicity across diverse open source projects.

Article Search Article: ase26main-p1633-p doi:10.1145/3832783.3837505

Generation of Web Apps with Agentic IDEs: An Empirical Assessment
Manuel Marceca, Maria Teresa Rossi, and Leonardo Mariani
(University of Milano-Bicocca, Italy; Gran Sasso Science Institute, Italy)
Agentic IDEs are among the most significant innovations in software engineering, aiming to accelerate application development through LLM-based agents that can assist developers during development. However, their evaluation in end-to-end development tasks involving the generation of complete applications remains limited. To fill this gap, we propose a rigorous comparative analysis of three popular agentic IDEs (Copilot, Cursor, and Windsurf) in the generation of five full-stack Web applications from scratch. Results show high maturity in the generation of established patterns, such as CRUD operations and authentication features. In contrast, the generation of less common distributed architectures, such as a task queue architecture, produces significantly more errors. Overall, results show that Agentic IDEs cannot replace developers but shift their role toward building software by orchestrating LLM-based agents through natural-language instructions and iterative refinement. Yet, each agentic IDE shows its peculiarities, although differences are narrow.

Article Search Article: ase26main-p1651-p doi:10.1145/3832783.3837506

Looks Good, but Does It Run? A Large-Scale Empirical Study of the Executability of Example Code in Model Cards
Simeng Yao, Jialin Zhao, Yang Zhang, Tun Li, Tao Wang, Changrong Xie, Zezhou Tang, and Yiwen Wu
(National University of Defense Technology, China)
Model cards document how machine learning models should be understood, evaluated, and reused in practice. Many also provide example code to help users access and use models, yet the executability of such code remains understudied. We take Hugging Face as an open model repository and conduct an empirical study of model-card example code. Among 291 model cards, 68.38% of the example-code snippets fail to run, suggesting that execution failures are common and may hinder model reuse. To characterize these failures, we construct a taxonomy comprising fifteen error categories and twenty-four subcategories and distill fifty reusable repair strategies. Based on these findings, we develop CardCodeFixer, a tool for automated repair. Experimental results show that CardCodeFixer increases executability from 29.63% to 56.08% with GPT-4o and from 31.75% to 51.85% with DeepSeek-v3. These results characterize execution problems in model-card example code and show that automated repair can improve executability.

Article Search Artifacts Available Article: ase26main-p1672-p doi:10.1145/3832783.3837507

From Thread to Pool: Modeling and Static Detection of Thread Pool Misuses in Java Programs
Baoquan Cui, Yucheng Qiu, Rui Han, Hang Gao, and Jian Zhang
(Institute of Software at Chinese Academy of Sciences, Beijing, China; Hangzhou Institute for Advanced Study at University of Chinese Academy of Sciences, Hangzhou, China; University of Chinese Academy of Sciences, China)
Asynchronous programming is fundamental to modern software performance, and Java ExecutorService-based executors have become a standard abstraction for managing concurrent tasks. While existing research has extensively studied traditional concurrency bugs, management misuses of executors remain under-analyzed. These defects arise not from the business logic of asynchronous tasks, but from how executors are configured, used, and terminated.
We present a taxonomy of seven executor-management misuse patterns and formalize them using a resource-management model. Based on this, we develop PoolGuard, a static analysis framework for detecting both executor-generic lifecycle/result-handling defects and ThreadPoolExecutor-specific configuration misuses. Extensive evaluation on a benchmark suite with real-world defects shows that PoolGuard is better than specialized async detectors as well as general-purpose analyzers such as SpotBugs and Infer. Applying PoolGuard to large-scale open-source systems, we identified 50 previously unknown issues, 24 of which have already been fixed or confirmed by developers, demonstrating the practical value of semantic analysis for executor-management defects.

Article Search Article: ase26main-p1687-p doi:10.1145/3832783.3837508

HERO: Hypothesis-Centered Root-Cause Analysis for Microservice Incidents
Jiewei Lyu, Junquan Yi, Shu Liang, Pengfei Chen, and Long Pan
(Sun Yat-sen University, China; Tencent, China)
Root-cause analysis (RCA) in cloud-native microservice systems is challenging because it requires integrating heterogeneous observability signals and deciding what to inspect next under limited diagnosis budgets. Existing large language model (LLM)-based RCA agents either rely heavily on predefined workflows, whose effectiveness is constrained by incomplete Standard Operating Procedure (SOP) coverage, or adopt open-ended diagnosis, which remains brittle when multiple plausible root-cause hypotheses are consistent with the observed evidence. We present Hero, a hypothesis-centered RCA framework that formulates diagnosis as controlled discrimination among competing hypotheses via bounded hypothesis-space instantiation, explicit belief-state tracking, and ambiguity-guided evidence acquisition. We evaluate Hero on OpenRCA and a real-world industrial incident dataset against representative workflow-based and open-ended baselines. On OpenRCA with DeepSeek-V3, Hero achieves an S_R of 79.77, outperforming the strongest ReAct-style baseline by 23.20 points while using 1.04 fewer actions on average. On the industrial dataset, it reaches an S_R of 85.00 and remains the best-performing method. These results suggest that explicitly controlling competing hypotheses is an effective and practically promising abstraction for microservice RCA, especially in real-world, production-style incidents with incomplete workflow coverage and high diagnostic ambiguity.

Article Search Article: ase26main-p1707-p doi:10.1145/3832783.3837509

Gray-Box Fuzzing in Local Space
Martin Jonáš, Jan Strejček, and Marek Trtík
(Masaryk University, Czech Republic)
This paper formally defines and tackles a key problem of automated test generation. The coverage problem is specified by a program execution path leading to a certain branching point and an input vector of this execution. The goal is to modify the input vector such that the program running on the modified input follows the same path but the branching point is evaluated to the opposite truth value. Standard fuzzing techniques use bit flips, random fuzzing, and gradient descent to solve this problem. We present an approach that uses numerical methods and linear algebra to transform the problem to so-called local space and applies the mentioned techniques there. An experimental evaluation on the Test-Comp 2025 benchmarks shows some improvements over the standard fuzzing techniques, both for the studied coverage problem and for the problem of test generation for maximal branch coverage.

Article Search Artifacts Available Article: ase26main-p1719-p doi:10.1145/3832783.3837510

Revisiting LLMs on New Feature Implementation in Real-World Software Development Practices
Jia Li, Tingxuan Huang, Tiankuo Zhao, Yiming Wang, Yiran Zhang, Anshun Hu, Kechi Zhang, Lecheng Wang, Zinan Sheng, Kui Liu, Ge Li, and Zhi Jin
(Wuhan University, China; Nanyang Technological University, Singapore; Peking University, China; Huawei Technologies, China)
Implementing new features in existing repositories is a key activity in software development. With the rapid advances in large language models (LLMs), benchmarking their capability in this task has attracted growing attention. However, existing benchmarks deviate from real-world practices in two aspects. First, they provide LLMs with unrealistic information such as target code signatures and file paths, which developers do not have access to in practice. Second, they only evaluate the final generated code using functional tests, without examining intermediate steps like requirement understanding and implementation planning. These limitations make it difficult to faithfully assess LLMs’ actual capabilities and diagnose their weaknesses.
To address the above limitations, we conduct a systematic evaluation to revisit the capability of large language models (LLMs) in new feature implementation, with a task setup strictly aligned with real-world software development practices. Our evaluation is built on NFEI, a curated benchmark sourced from merged feature pull requests of active GitHub repositories, where each target feature involves cross-snippet multiple code modifications, and LLMs are only provided with the feature description and the target repository throughout the evaluation. Beyond the realistic input settings, we further annotate fine-grained labels for the intermediate steps of the development workflow, including requirement clarification and implementation planning. This enables our evaluation to support a comprehensive stage-wise assessment across three granularities with seven dedicated metrics. We evaluate seven popular LLMs under two development paradigms and three generation strategies. Results show that new feature implementation remains highly challenging, with an overall average Test Pass Rate of only 29.84%. Our stage-wise analysis reveals that intermediate step quality is closely linked to final code performance, and injecting ground-truth intermediate artifacts yields an average absolute gain of about 18%, yet performance remains limited. Qualitative error analysis further identifies three recurrent failure patterns, suggesting that new feature implementation of LLMs adhering to real-world world software development practices remains challenges.

Article Search Article: ase26main-p1768-p doi:10.1145/3832783.3837511

Self-Discovering Security Oracles: Meta-learning Vulnerability Detection Strategies through Adversarial Self-Play
Md Rashedul Hasan and Hamid Bagheri
(University of Nebraska-Lincoln, USA)
Current Large Language Model (LLM) based vulnerability detection systems rely on static prompting strategies and fixed analysis pipelines that fail to adapt to evolving threat landscapes. These approaches struggle to discover unknown vulnerability patterns beyond their training distributions, limiting their effectiveness against emerging security threats. This paper explores adversarial meta-learning where foundation models autonomously discover vulnerability detection strategies through reinforcement learning and competitive self-play. We present MetaMorphSES, a framework that enables Foundational models to evolve in terms of both vulnerability generation capabilities (as a red team) and detection strategies (as a blue team) through adversarial co-evolution without predetermined prompts or fixed architectures. The system learns to generate increasingly sophisticated synthetic vulnerabilities while simultaneously developing detection strategies that generalize to real-world security flaws. Through a dual-loop optimization process, the inner loop generates and detects vulnerabilities while the outer meta-learning loop discovers optimal detection strategies through reinforcement learning. Our experiments on six real-world vulnerability corpora demonstrate that meta-learned security strategies yield substantially stronger detection of unseen vulnerabilities (up to 0.90 F1) against nine strong baselines spanning rule-based static analysis, zero-shot and fine-tuned LLMs, and recent vulnerability detectors, with the system reaching a stable equilibrium. While training remains resource-intensive, the approach establishes adversarial meta-learning as a novel and effective way to discover security vulnerabilities with foundational models through self-directed adaptation without explicit programming of detection rules.

Article Search Artifacts Available Article: ase26main-p1822-p doi:10.1145/3832783.3837512

S2CAgent: A Self-Learning Agent Knowledge Graph Framework for Software Supply Chain Analysis
Youfei Huang, Zhong Li, Minxue Pan, and Xuandong Li
(Nanjing University, China)
The extensive reuse of open-source software (OSS) accelerates development. However, it also creates increasingly complex software supply chains (SSCs) and intensifies the attacker-defender asymmetry, making attacks "low-cost, high-impact, and hard-to-detect." Software Bill of Materials (SBOMs) and Software Composition Analysis (SCA) are widely used for SSC analysis, but both have clear limitations. SBOMs frequently lack vulnerability information, while SCA tools are typically limited to specific ecosystems. To address these limitations, we first construct a multi-source software supply chain knowledge graph (SSCKG) that unifies heterogeneous information and supports continuous evolution through automated updates. Building on this foundation, we propose S2CAgent, a neural-symbolic large language model (LLM) agent framework for SSC analysis. S2CAgent supports natural language questions by inducing symbolic reasoning rules from the SSCKG. These rules align complex questions with underlying graph semantics and multi-hop reasoning patterns. Furthermore, the agent leverages structured evidence from the SSCKG and unstructured web knowledge, thereby mitigating LLM hallucinations and alleviating knowledge incompleteness. In addition, it incorporates a self-learning mechanism that synthesizes and refines tool-augmented trajectories to improve future reasoning. We evaluate S2CAgent on a newly constructed cross-ecosystem benchmark that includes three core tasks: vulnerability impact assessment, software dependency analysis, and license compliance checking. Experimental results show that S2CAgent outperforms baselines across multiple metrics and remains stable under complex multi-hop reasoning. In particular, S2CAgent has been deployed at a leading ICT corporation, where it effectively handles complex real-world analysis cases, further demonstrating its practical utility.

Article Search Article: ase26main-p1823-p doi:10.1145/3832783.3837513

ArkEval: Benchmarking and Evaluating Automated Code Repair for ArkTS
Bang Xie, Senjian Zhang, Zhiyuan Peng, Wei Chen, Xin Yin, Chenhao Ying, and Yuan Luo
(Shanghai Jiao Tong University, China; Zhejiang University, China)
Automated program repair benchmarks largely cover mature languages such as Python and Java, leaving limited support for ArkTS and OpenHarmony. We present ArkEval, an executable repository-level benchmark for evaluating repair on real ArkTS/OpenHarmony issue-resolution tasks. To the best of our knowledge, it is the first benchmark with this scope.
We mined more than 6,000 pull-request and merge-request records available through July 3, 2025, from nine public repositories. Automated profiling and unanimous screening by three ArkTS/OpenHarmony developers produced 502 instances. Their reproduction tests were constructed through a six-round benchmark-wide refinement process, followed by targeted closure for the remaining six instances. We evaluated eight current models with the same localization-guided, RAG-off repair protocol. The best model compiled 54.38% of its patches, whereas the highest behavioral Pass@1 was 19.32%. ArkEval therefore separates build validity from behavioral repair and supplies traceable tests for studying both.

Article Search Article: ase26main-p1862-p doi:10.1145/3832783.3837514

From Vibe to Spec: An Empirical Study to Demystify Specifications in Code Generation
Ruirui He, Hanqing Lu, Lezhi Ma, Kai Ye, Qiong Wu, Shangqing Liu, Lei Bu, and Xuandong Li
(Nanjing University, China)
Recent advances in large language models (LLMs) for code generation have given rise to vibe coding, a programming practice in which developers interact with LLMs through unstructured natural language and primarily validate implementations by inspecting overall execution outputs. However, the inherent ambiguity and inconsistency of natural language present substantial challenges for generating complex and large-scale software systems. To mitigate these limitations, an emerging AI programming paradigm—specification-driven development—advocates providing LLMs with precise and consistent specifications to guide code generation. Despite its promise, fundamental questions regarding the appropriate forms of specifications and the granularity at which they should be supplied to LLMs remain largely unexplored.
To address this gap, we present the first empirical study that systematically investigates the impact of two orthogonal dimensions of specifications—form and granularity—on the effectiveness of LLM-based code generation. We construct a dataset comprising five medium-sized software projects, ranging from several hundred to over one thousand lines of code, implemented in multiple programming languages and spanning diverse application domains (e.g., system utilities and games). For each project, we design specifications in three forms—natural language, coarse-grained pseudocode, and fine-grained pseudocode—to capture the form dimension. To examine granularity, we further decompose the fine-grained pseudocode into four levels of input granularity: line-level, function-level, module-level, and project-level. Using this dataset, we evaluate code generated by three representative models, GPT-5, DeepSeek-V3.1, and Qwen-3.7. Evaluation is conducted using a comprehensive set of metrics, including text-similarity-based metrics (e.g., BLEU-4 and CodeBLEU), test-based metrics (e.g., accuracy), and human-centered metrics (e.g., usability and logical consistency).
Our results demonstrate that using fine-grained, structured pseudocode specifications significantly enhances generation stability and quality. Regarding granularity, while coarser-grained specifications (project and module level) generally achieve higher code quality, organizing specifications at the module level provides the best overall balance between quality, stability, and token efficiency. Moreover, additional analyses indicate that this module-level granularity yields the most user-friendly outcome for human developers.

Article Search Article: ase26main-p1876-p doi:10.1145/3832783.3837515

Instruction Alignment for Binary Code Representation Learning
Huaijin Wang and Shuai Wang
(Shandong University, China; Hong Kong University of Science and Technology, Hong Kong)
Binary code representation learning is a fundamental problem in software security and reverse engineering. Existing methods mainly learn function-level embeddings that capture coarse-grained semantic relationships between binary functions, but they largely ignore fine-grained instruction-level correspondences. This limitation misses valuable supervision signals available from compiler debug information, which can support the learning of more accurate and interpretable binary code representations.
We propose to leverage instruction alignment knowledge to further improve binary code representation learning. Our preliminary study reveals that models finetuned for function-level binary code similarity exhibit substantially better instruction alignment than their pre-trained model, suggesting a strong correlation between instruction alignment and function-level embedding quality. Motivated by this observation, we design a training approach that explicitly incorporates instruction alignment as an auxiliary training objective. Our experiments show that instruction alignment training improves retrieval accuracy and provides more discriminative signal for the model's similarity judgments.

Article Search Article: ase26main-p1967-p doi:10.1145/3832783.3837516

Formula Completion Suggestions for Alloy Models with Selection Guidance
Mohammad Nurullah Patwary and Allison Sullivan
(University of Texas at Arlington, USA)
Alloy is a declarative modeling language that is well-suited for verifying system designs. Unfortunately, writing correct Alloy models is a difficult task due to Alloy’s expressive operators, which allow for succinct formulations of complex properties, but can be hard to reason over manually. Case in point, a recent empirical study found that 75% of the time, novice Alloy users will write formulas that are incorrect or fail to compile. To aid developers in writing valid formulas, we outline a framework in which the user is presented with relevant completion suggestions as they are constructing their formulas. Moreover, using Alloy itself, we provide novel automated guidance that helps users select the completion that best matches their intention. Our evaluation highlights that there is nominal overhead to create these suggestions, our ranking strategy is effective, and our automated guidance can balance providing context with reducing the amount of information we present.

Article Search Article: ase26main-p1981-p doi:10.1145/3832783.3837517

COBug: A Bug Localization Pipeline for COBOL: Does It Work? An Exploratory Study
Kavyasri Gajula, Phanindra Kodela, and Sridhar Chimalakonda
(Indian Institute of Technology Tirupati, India)
COBOL still runs core banking, insurance, and government systems, but the developers who understand it are retiring faster than they are replaced, and defects in this code are costly to diagnose. Bug localization could help, yet the technique has matured on Java and Python, while COBOL lacks the standardized bug reports, faulty revisions, and validated report-to-file links required for retrieval-based evaluation.
We present COBug, a leakage-controlled benchmark and evaluation pipeline for file-level bug localization in COBOL: 552 deterministic queries across 12 X-COBOL repositories, each carrying one isolated source mutation with mechanically traceable ground truth and a symptom-focused report generated without reading the target source file. Using COBug, we evaluate lexical retrieval, pointwise re-ranking, LambdaMART, and frozen CodeBERT under leave-one-repository-out evaluation. None reliably outperforms the repository-matched random floor. After Holm correction, no method–baseline comparison is statistically significant. Lexical retrieval comes closest at MAP 0.1508 against a floor of 0.1229, though two repositories holding 15 queries contribute over 80% of that margin.
We argue that this may partly reflect the benchmark construction itself: a report synthesized in strict isolation from its target may carry limited information about which file is faulty. We therefore present the result as indicating a possible limitation of this template-based construction, rather than as a verdict on COBOL bug localization, and release the pipeline with its manifests and provenance records.

Article Search Article: ase26main-p1990-p doi:10.1145/3832783.3837518

Accurate and Low-Cost Residual Risk Assessment via Sampled Profiling and Structure-Aware Coverage Amplification
Seongmin Lee, Işıl Özgü, Marcel Böhme, and Miryung Kim
(University of California at Los Angeles, USA; MPI-SP, Germany)
Estimating the residual risk of undiscovered bugs—and thus when to stop fuzzing—is critical for determining testing adequacy. Traditionally, this assessment relies on coverage instrumentation, which records complete coverage of every execution but imposes a 60-70% performance slowdown, reducing fuzzing throughput and delaying bug discovery. This overhead raises a fundamental question: can partial observation provide high-fidelity confidence at a fraction of the coverage instrumentation cost?
We propose a risk assessment method in the context of black-box fuzzing that leverages hardware-enabled performance profiling—replacing coverage instrumentation with instruction pointer (IP) sampling—as the core mechanism for low-overhead observation. To recover the accuracy inevitably lost to the sparse nature of sampling, we employ must-execute analysis, a static control-flow analysis that identifies all statements and basic blocks that must have executed to reach, or must execute after, an observed sample, effectively reconstructing missing execution paths from partial observations. Despite the partial coverage inherent to profiler-based sampling, our risk estimation method achieves 4.27× higher throughput than complete-coverage instrumentation, enabling fuzzing campaigns to reach stopping thresholds 2× faster while maintaining high accuracy (1.73 average relative error compared to the ground truth). Ultimately, this efficiency translates into superior bug discovery, triggering more bugs with speedups of up to 10× within the same time budget.

Article Search Article: ase26main-p2006-p doi:10.1145/3832783.3837519

SPSYN: Synthesizing DeFi Price-Manipulation Exploits via Semantic Recovery and State-Guided Search
Bosi Zhang, Ningyu He, Guangdong Bai, and Haoyu Wang
(Huazhong University of Science and Technology, China; Hong Kong Polytechnic University, Hong Kong; City University of Hong Kong, Hong Kong)
Price manipulation is one of the most damaging vulnerability classes in decentralized finance (DeFi) protocols. Existing approaches mainly stop at warning generation or rely on rigid attack templates, and therefore struggle to determine whether a victim protocol is profitably exploitable under a concrete chain state. This problem is challenging because the attack-relevant victim-side steps are highly protocol-specific, while successful exploitation further depends on brittle runtime conditions such as reserve states, token-specific guards, and narrow numeric ratios. In this paper, we present SPSYN, a DeFi price-manipulation exploit synthesizer that combines semantic recovery with state-guided search. SPSYN first extracts structured cross-contract contexts and uses an LLM to recover attack-relevant victim-side function roles. It then organizes the recovered functions into stage-consistent candidates and scenario-guided initial seeds. Finally, it synthesizes exploit witnesses on a concrete chain snapshot through state-guided search under a semantic-aware state machine. Runtime observations reveal whether the transformed sequence has established a meaningful anomaly, whether it should continue to be expanded, and how later mutations should be refined. In this design, semantic recovery narrows the search to the right victim-side actions, while exploitability is validated through actual execution on the target snapshot. On 29 historical incidents, SPSYN synthesizes profitable exploit witnesses for 27 cases (93.1%), outperforming ItyFuzz (11/29), CPMMX (19/29), and FlashSyn (7/29), with a median time-to-first witness of 114 seconds on successful cases. During on-chain deployment across Ethereum and Binance Smart Chain, SPSYN further discovers 9 previously unreported profitable vulnerabilities with a cumulative potential profit of $35.1K.

Article Search Article: ase26main-p2062-p doi:10.1145/3832783.3837520

To Think or Not to Think: Evaluating LLM Reasoning and Agents in Vulnerability Detection
Hua Beng Tan, Ratnadira Widyasari, Merve Astekin, Arda Goknil, Hasan Sözer, Yan Naing Tun, Erik Johannes Husom, Lwin Khin Shar, and Lingxiao Jiang
(Singapore Management University, Singapore; SINTEF, Norway; Ozyegin University, Türkiye)
Large language models (LLMs) are increasingly studied for software vulnerability detection, but the benefits of recent reasoning-enabled (“thinking”) modes remain unclear. At the same time, agentic LLM designs aim to enhance analysis through structured collaboration, while practical software security workflows require explanations that facilitate inspection, validation, and remediation. Because prior studies often examine these dimensions separately or under heterogeneous settings, their individual effects on vulnerability detection remain difficult to disentangle. This paper addresses the gap through a phased, factor-separated empirical study on the PrimeVul Pair benchmark. In the first phase, we compare instruct and thinking variants of four open models under zero- and few-shot prompting without agents. In the second phase, we evaluate single-, dual-, and multi-agent (SA, DA, and MA) designs alongside call-matched single-agent baselines, alert-burden metrics, and cost analysis. In the third, we assess explanations through blinded human and calibrated LLM evaluation across correct and incorrect predictions, controlling for response length and model family. Thinking improves non-agent (NA) F1 in 7 of 8 settings, mainly through recall, while DA achieves the highest F1/recall but flags 88.2% of functions as vulnerable. MA performs better on pairwise discrimination, although call-matched self-revision achieves the best Pair-wise Correct Prediction (P-C) overall. Thinking also uses 2.7–4.4× more energy, and its F1 benefit declines with greater orchestration. Explanation gains are model-specific, partly length-mediated, and do not distinguish correct from incorrect predictions.

Article Search Article: ase26main-p2084-p doi:10.1145/3832783.3837521

Demystifying Solana Bots: From GitHub Blueprints to On-Chain Fingerprints
Xiaoye Zheng, Yujing Chen, Minghao Wu, David Lo, Difan Xie, Daoyuan Wu, Xiaohu Yang, and Zhiyuan Wan
(Zhejiang University, China; Singapore Management University, Singapore; Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security, China; Lingnan University, Hong Kong)
Solana is an emerging blockchain platform designed for high throughput and low transaction fees, making it inexpensive to submit transactions at scale and, consequently, increasing exposure to bot spamming and related financial exploitation. Solana bots are typically off-chain software systems that operate in a competitive on-chain execution environment by constructing and submitting transactions, and the bot-related transactions on the decentralized exchanges exceed 250 million dollars in daily trading volume in January 2026. Prior studies on Solana have examined system performance, smart-contract security, and specific on-chain phenomena. However, we still lack a systematic understanding of what Solana bots implement in practice and how these implementations manifest as observable on-chain execution fingerprints. To address this gap, we performed a large-scale empirical study of Solana bots from two complementary views: (i) 586 bot repositories collected from GitHub, and (ii) 200 bot addresses on Solana, with over 44 million on-chain transactions. Our study derives an implementation-grounded taxonomy of Solana bots comprising 15 categories grouped into five domains (e.g., Trading Operations, MEV, and On-chain Analytics), identifies a largely shared five-stage operational pipeline manifested in bot implementations, and uncovers systematic variation in on-chain trading behaviors of Solana bots across diverse trading platforms and assets. Based on our findings, we highlight future research directions, and provide recommendations for building and operating bots on the Solana blockchain.

Article Search Article: ase26main-p2100-p doi:10.1145/3832783.3837522

CLEAR: Causal Context-Based Agentic Reasoning for Vulnerability Detection
Sungju Yun, Sijune Hwang, Yeonjoon Lee, Kyungtae Kang, and Sungbin Park
(Hanyang University, Republic of Korea)
Detecting source code vulnerabilities is increasingly difficult as modern security flaws are rooted in complex causal dependencies between execution flows, control conditions, and program states. Despite recent advances in Large Language Models (LLMs) and multi-agent frameworks, existing approaches primarily address superficial similarities between benign and vulnerable functions while failing to capture the complex causal dependencies inherent in security flaws. To address these limitations, we propose Causal Context-based Agentic Reasoning (CLEAR), a novel multi-agent vulnerability detection framework integrated with a causal knowledge graph. CLEAR systematically constructs a Vulnerability Causal Knowledge Graph (VCKG) that models the causal chains between entrypoints, preconditions, root causes, and fix intents across vulnerability instances. Leveraging this structured knowledge, four specialized agents including the Collector, Claim, Critic, and Judge collaboratively verify vulnerability hypotheses through retrieved causal contexts. Experimental results on C/C++ and Java vulnerability benchmarks demonstrate that CLEAR improves Pair-Correct (P-C) performance by 130.7% and 71.56% over state-of-the-art approaches, respectively, demonstrating the effectiveness of causal knowledge graph–guided reasoning for automated vulnerability detection.

Article Search Article: ase26main-p2153-p doi:10.1145/3832783.3837523

Large Language Model Enabled Symbolic Execution for Automated Functional Analysis
Meixi Liu, Zhenbang Chen, Xudong Wang, Ziran He, Jiachen Gu, Minghui Chen, Lei Wang, and Wei Dong
(National University of Defense Technology, China)
Symbolic execution shows promise in automated program analysis by systematically exploring the path space of the program. However, due to the inherent limitations of program analysis techniques, symbolic execution cannot understand the program’s functionality, which hinders its application in functional analysis and defect detection. This paper introduces FuSE, a method that enhances symbolic execution with Large Language Models (LLMs) to automatically annotate program paths with functional specifications. A program path is annotated with a natural language description of its functionality, a precondition, and a postcondition. Our method significantly reduces the effort required for functional testing and defect detection. Based on UnitTestBot, FuSE can successfully annotate functional specifications for C, C++, Java, and Python programs, with success rates of 98.2%, 95.52%, 87%, and 95.37%, respectively. In our user study with professional developers and testers, FuSE helps users identify 21.67% more functional defects and improves functional coverage by 15.83%, demonstrating its practical utility.

Article Search Artifacts Available Article: ase26main-p2155-p doi:10.1145/3832783.3837524

Structure-Guided Semantic Deobfuscation of Android Identifiers
Mingyang Chen, Zhentao Xie, Yaqi Gao, Shuang Li, and Wenrui Diao
(Shandong University, China)
Android developers frequently employ identifier renaming to obfuscate code and hinder reverse engineering. This practice effectively conceals program semantics. However, it severely impedes downstream security analysis tasks, particularly manual inspection of malicious and privacy-sensitive behaviors. Existing deobfuscation techniques rely on code similarity, learned naming priors, or local LLM reasoning. These methods often exhibit severe limitations when applied to proprietary, heavily obfuscated, or cross-file context-dependent code. In this paper, we present DeMask, a novel framework that integrates static program analysis with Large Language Models (LLMs) to recover meaningful identifiers from obfuscated Android apps. Guided by program structure, DeMask executes a bottom-up, context-aware inference process for whole-app semantic reconstruction. It incorporates tailored naming constraints and optimization strategies to mitigate LLM hallucinations and ensure scalability for real-world codebases. We implemented a prototype and evaluated it on diverse datasets of benign and malicious Android apps. The results demonstrate that DeMask significantly outperforms prior work in semantic accuracy and naming consistency. Specifically, it achieves an average effective semantic recovery of over 88.6% across all evaluation settings.

Article Search Article: ase26main-p2252-p doi:10.1145/3832783.3837525

Orbis: Guiding Symbolic Execution Techniques to Maximize Option-Related Branch Coverage
Minjong Kim, Sungjae Hwang, and Sooyoung Cha
(Sungkyunkwan University, Republic of Korea)
We present Orbis, a complementary approach that guides symbolic execution techniques toward maximizing option-related branch (OB) coverage. Although state-of-the-art symbolic execution techniques have substantially improved code coverage and bug detection, existing techniques often fail to exercise core program functionalities, which are frequently exposed as program options. To quantify this limitation, we evaluate how effectively current techniques cover option-related branches and find that they reach only 8.0% of the targeted OBs, which are accessible only through specific options. Orbis overcomes this challenge by systematically guiding symbolic execution to improve OB coverage. For a given program, Orbis automatically identifies all defined options and determines their associated branches by analyzing dynamic execution traces. It then iteratively constructs target option arguments by stochastically combining identified options according to their estimated potential, directs symbolic execution to explore states influenced by these arguments, and refines their potential while simultaneously reducing SMT-solver invocations using data accumulated during symbolic execution. Experimental results on 15 open-source C programs show that integrating Orbis with four independent state-of-the-art techniques significantly increases OB coverage to 323.0%, boosts total branch coverage by 112.1%, and detects five additional unique bugs on average compared to the standalone techniques.

Article Search Article: ase26main-p2260-p doi:10.1145/3832783.3837526

MultiFixer: A Coordinator-Proposer Based Multi-agent Framework for Fixing Multi-hunk Bugs
Haichuan Hu, Chunrong Fang, Ye Shang, Jiawei Liu, Weifeng Sun, Guoqing Xie, Chenxing Zhong, and Quanjun Zhang
(Nanjing University of Science and Technology, China; Nanjing University, China; Singapore Management University, Singapore)
Automated Program Repair (APR) has benefited greatly from Large Language Models (LLMs), but existing LLM-based APR methods still struggle with multi-hunk bugs that require coordinated changes across multiple locations. These bugs demand repository-level context understanding, repair-order scheduling, and effective hunk-level patch generation and selection. To address these challenges, we propose MultiFixer, a novel Coordinator-Proposer based multi-agent framework for multi-hunk repair. MultiFixer performs tool-augmented bug analysis, constructs fine-grained repair context, iteratively generates patches through a Coordinator-Proposer architecture, and applies two-stage patch refinement for syntactic and semantic correctness. We evaluate MultiFixer on 835 bugs from Defects4J and three vulnerability benchmarks. On Defects4J, MultiFixer fixes 326 bugs, including 62 multi-method and 27 multi-file bugs, and outperforms prior APR baselines in the reported comparisons with the same base model. Moreover, MultiFixer also fixes 46 multi-hunk bugs among 95 unique fixes. When combined with Claude-3.5-Sonnet, MultiFixer repairs 420 bugs, establishing a new state of the art on Defects4J. On VUL4J, MultiFixer repairs 24 real-world vulnerabilities, including 5 multi-hunk cases. On the multi-hunk subsets of SEC-bench and PatchEval, MultiFixer fixes 11 and 19 vulnerabilities, respectively, outperforming all compared baselines under GPT-3.5. These results demonstrate the effectiveness of MultiFixer for multi-hunk repair.

Article Search Article: ase26main-p2372-p doi:10.1145/3832783.3837527

Mining Tactics for Automated Theorem Proving
Jian Fang, Yixun Yao, and Yingfei Xiong
(Peking University, China)
Automated theorem proving is essential for the formal verification of safety-critical systems. As the corpus of formal proofs grows, a natural paradigm is to learn from existing proofs. However, current learning-based approaches predominantly train Large Language Models (LLMs) as end-to-end provers, which yields resource-intensive, opaque systems. Conversely, while traditional symbolic provers are computationally efficient, how to automatically improve these solvers from data remains an open challenge.
This paper bridges this gap by proposing LLM2Ltac, a novel approach that leverages the reasoning power of LLMs not as end-to-end provers, but as intelligent synthesizers to mine purely symbolic tactics from data. Given a corpus of formal proofs, LLM2Ltac asks an LLM to identify latent proof strategies and formalize them into reusable tactics. These tactics are verified for validity and generalizability, and finally integrated into symbolic provers to enhance their automated proving capabilities without the runtime cost of LLMs.
We implement LLM2Ltac on Rocq 8.20.0 and mine tactics from 11,725 theorems in the standard library. We evaluate our approach on 6,199 theorems from four large real-world verification projects, namely, CompCert, Coq-Art, Ext-Lib, and VFA. Results show that the mined tactics improve CoqHammer to prove 23.87% more theorems, and when integrating the improved CoqHammer with Claude Code, the overall proved theorems increases by 9.90%, indicating the effectiveness of LLM2Ltac.

Article Search Article: ase26main-p2413-p doi:10.1145/3832783.3837528

Leveraging System-Level Observations to Inform Bayesian Learning of Model Parameters for Quantitative Verification
Simos Gerasimou and Xingyu Zhao
(Cyprus University of Technology, Cyprus; Wuhan University, China)
Combining Bayesian learning and quantitative verification is a powerful toolset for analysing key quantitative properties of software systems, like reliability and response time. However, the accuracy and robustness of verification results strongly depend on the prior knowledge (PK) underlying Bayesian inference. This knowledge reflects original beliefs about the probability of events and typically depends on domain expertise. Using inaccurate or uninformative PK can negatively affect quantitative analysis, yielding incorrect verification results. Our EPIK approach tackles this important challenge by eliciting and embedding PK in quantitative verification equipped with Bayesian estimators. Unlike existing approaches that require PK on formal model transition parameters, EPIK leverages system-level properties that are directly observable and are linked to real-world semantics. EPIK formulates a twofold optimisation problem to derive the distributions of unknown transition parameters and then embeds these distributions to verify new or difficult-to-measure (elusive) properties. The detailed experimental evaluation using multiple variants of real-world case studies and diverse EPIK instantiations shows its effectiveness, flexibility and generality.

Article Search Article: ase26main-p2427-p doi:10.1145/3832783.3837529

Generating Loop Invariants for Deductive Verification using Iterator Profiles
Keisuke Nishimura, Jean-Pierre Lozi, and Julia Lawall
(Inria, France)
Recent advances in deductive verification have gradually made it practical to verify the functional correctness of real-world software. However, creating loop invariants still requires manual effort. We propose to reduce this burden by automatically generating invariants. In the context of generating invariants for real-world software, the key requirements are to support diverse index iteration patterns and to capture precise information about loop behaviors.
In this paper, we introduce the idea of an iterator profile: a loop-invariant template for an iteration pattern. Once a user writes a profile, it can be reused for any iterator-based loop that shares the same iteration pattern. We present LoopAnnotator, a tool that generates loop invariants for iterator-based loops by combining an iterator profile with the result of automatic program analysis of the loop body. LoopAnnotator successfully generates invariants for more than 70

Article Search Article: ase26main-p2432-p doi:10.1145/3832783.3837530

RustGTC: Constraint-Guided Test Generation for Rust APIs via Comment Analysis and Code Context
Yichi Zhang, Jinhao Huang, and Yang Feng
(Peking University, China; Institute of Software at Chinese Academy of Sciences, China; Nanjing University, China)
Manually crafting test cases for Rust APIs is labor-intensive and error-prone, especially because of Rust’s advanced features and complex trait system. Recent advances in large language models (LLMs) have shown promise in automated test generation; however, existing approaches often underuse documented input requirements embedded in API comments. We present RustGTC, a framework that uses API comments to guide Rust API test generation. RustGTC extracts documented parameter constraints from comments, constructs tests around boundary cases, and repairs generated tests to improve compilability. We evaluate RustGTC on 12 Rust crates, achieving high coverage and uncovering 41 defects. Additionally, we submitted 132 test cases; 123 were accepted, 9 were rejected. These results show that RustGTC uses documented constraints to guide Rust API test generation and reveal comment-code inconsistencies around boundary cases.

Article Search Article: ase26main-p2466-p doi:10.1145/3832783.3837531

HyBinMAS: LLM-Powered Hybrid Analysis for Function Name Recovery of Stripped Binaries
Chenxi Wang, Lei Xu, Xincheng He, Gaozhan Yu, and Ye Wang
(Nanjing University, Nanjing, China; Jiangnan University, China; Southeast University, China)
Binary function name recovery is vital for understanding stripped binaries, yet existing learning-based methods face significant hurdles. Current approaches often struggle to capture inter-procedural context and complex control flows, leading to semantic ambiguity. Furthermore, their reliance on static analysis and fixed training distributions severely limits their generalization across heterogeneous architectures and unseen binaries. In this paper, we propose a novel approach HyBinMAS that reformulates binary function name recovery as an evidence-driven semantic-state update process with LLM-Powered hybrid analysis. For each target function, HyBinMAS maintains a semantic state ⟨ name, summary ⟩ and iteratively refines it using static evidence, inter-procedural semantic propagation, and dynamic evidence. Specifically, it introduces an Attributed Call Graph (ACG) as a stateful semantic memory and incorporates runtime feedback to validate and correct semantic hypotheses. It further employs a collaborative multi-agent workflow, consisting of SemanticRestorer, TestGenerator, and ExecutionValidator to realize the state updates through semantic hypothesis generation, executable validation, and behavioral feedback interpretation. To address the lack of ARM-64 datasets, we construct an ARM-64 Benchmark comprising over 8,000 binary functions. Extensive experiments across five architectures demonstrate that HyBinMAS consistently outperforms existing approaches, achieving an average F1 score of 0.343 and improving the F1 score by 39.43% over the strongest baseline. Moreover, on a controlled subset, incorporating execution-grounded evidence yields an 18.14% F1 improvement over static-only analysis, demonstrating the complementary value of dynamic feedback for semantic-state update.

Article Search Article: ase26main-p2478-p doi:10.1145/3832783.3837532

Beyond Syntactic Differencing: Semantic-Aware Scheduling for Cross-Configuration Incremental Builds
Xiaolong Dai, Shanshan Li, Jun Lyu, and Xintao Niu
(Nanjing University, China)
Modern configurable software systems are routinely built and tested across thousands of configuration variants to ensure correctness under diverse settings. Reusing build artifacts across configurations is therefore crucial to reduce redundant effort, yet remains challenging. Existing approaches estimate rebuild cost using syntactic option differences and apply greedy scheduling, which often misestimate true costs and yield suboptimal build orders. We present SIBS, the first semantics-aware scheduling framework for incremental builds across configurations, addressing both inaccurate cost estimation and the limitations of greedy scheduling. The key idea is a semantic distance metric that models the impact of configuration changes on rebuild scope by capturing option-level effects on code generation, accounting for global configuration headers, and propagating changes through dependency graphs. This enables cost-aware global scheduling formulated as an Asymmetric Traveling Salesman Problem (ATSP).
Across 21 real-world systems and five independently sampled configuration batches, SIBS reduces observed build time relative to the state-of-the-art build scheduler in 17 systems. On average, it saves 196s (21%), with a median saving of 40s (3%), and achieves a 27.7% geometric-mean reduction in build time across projects. Batch-level reductions range from 25.7% to 29.3%, and the scheduling overhead remains modest (median 3.1s; mean 12.9s). These results demonstrate that semantics-aware cost modeling and global scheduling provide substantial and robust efficiency gains across diverse configurable systems.

Article Search Artifacts Available Article: ase26main-p2497-p doi:10.1145/3832783.3837533

Probe to Generate: Program Variant-Guided Test Augmentation for Repository-Level Repair Benchmarks
Chenglin Li, Yisen Xu, Zehao Wang, Shin Hwei Tan, and Tse-Hsun (Peter) Chen
(Concordia University, Canada)
Test-based benchmarks such as SWE-bench have become a standard basis for evaluating automated issue resolution agents, deeming a patch correct if it passes a provided regression test suite. In practice, weak test suites can admit plausible but semantically incorrect patches, inflating reported agent performance. We present PROBE, a test augmentation framework that uses semantically modified program variants as behavioral probes to identify and close gaps in benchmark test suites. Variants of the reference patch that survive the original tests reveal under-constrained behaviors, which then guide targeted regression test generation. Each generated test is retained only if it passes on the reference patch, fails on at least one surviving variant, and remains robust under behavior-preserving transformations. On SWE-bench Verified, 77% of instances admit at least one surviving variant. PROBE generates 1,014 validated tests across 211 instances, increasing patch-region line and branch coverage by 10.8 and 9.5 percentage points. Re-evaluating the top-10 repair agents with the augmented suites reduces resolved rates by 4.2%-9.0%, showing that many previously accepted patches exploit benchmark test gaps rather than fully satisfying the intended repair semantics. These findings demonstrate that benchmark evaluation is not solely a patch-generation problem but also a test-strength problem.

Article Search Article: ase26main-p2532-p doi:10.1145/3832783.3837534

Safeguarding LLM Agents from Misalignment through Provenance Analysis
Yining She, Yiliang Liang, and Eunsuk Kang
(Carnegie Mellon University, USA)
As LLM agents gain increasing access to powerful tools, ensuring that their actions align with the user’s intent becomes critical. When an agent’s proposed action deviates from that intent—a phenomenon called misalignment— it may cause harm that is difficult to undo. Existing runtime guardrails rely on an LLM-as-a-judge paradigm that lacks a systematic framework for reasoning about alignment, often producing inconsistent or difficult-to-audit judgments. Motivated by provenance analysis, we propose a conceptual framework that formalizes misalignment detection as determining whether a proposed tool call is supported by traceable evidence in the agent’s context. Based on this framework, we build ProvenanceGuard, a multi-stage pipeline that analyzes the agent’s action for three types of misalignment before its execution and only allows aligned actions. We evaluated ProvenanceGuard on AgentSafetyBench and WorkBench, across 11 backbone LLMs. Compared to the LLM-as-a-judge baseline, ProvenanceGuard reduces error rate on misaligned traces from 44.3

Article Search Article: ase26main-p2533-p doi:10.1145/3832783.3837535

PropGen: Automated Property Generation for Property-Based Testing of Mobile Apps
Yiheng Xiong, Shiwen Song, Bo Ma, Ting Su, and Xiaofei Xie
(Singapore Management University, Singapore; East China Normal University, China)
Mobile apps often suffer from functional bugs that do not cause crashes but instead manifest as incorrect behaviors under specific user interactions. Such bugs are difficult to detect by conventional automatic testing techniques because they often lack explicit test oracles. Property-based testing can effectively expose them by specifying intended behavior as properties and checking them under diverse interactions. However, its practical use is limited by the reliance on manually written properties, which are difficult and expensive to construct.
To address this limitation, this paper explores the use of large language models (LLMs) to automate property construction for property-based testing of mobile apps. This is challenging in two ways. First, it is difficult to systematically uncover and execute diverse app functionalities. Second, it is difficult to derive valid properties from functionality execution results. To address these challenges, we introduce PropGen, which infers candidate app functionalities as hypotheses from GUI states, validates each hypothesis by executing it to collect behavioral evidence, synthesizes properties from the collected evidence, and refines imprecise properties based on testing feedback. We implemented PropGen and evaluated it on 12 real-world Android apps. The results show that PropGen can effectively identify and execute app functionalities, generate valid properties, and refine most imprecise ones. Across all apps, PropGen inferred 1,210 valid functionalities and correctly executed 977 of them, compared with 491 and 187 for the baseline. It generated 985 properties, 912 of which were valid, and successfully refined 118 of 127 imprecise ones exposed during testing. Using the resulting properties, we found 25 previously unknown functional bugs, many of which were missed by existing testing techniques.

Article Search Article: ase26main-p2547-p doi:10.1145/3832783.3837536

From Function to Repository: Towards Understanding and an Agentic Approach for Repo-Level Directed Input Generation
Haoyu Wang, Ming Wen, Haoran Yan, Jiayi Huang, Xuanhua Shi, and Hai Jin
(Huazhong University of Science and Technology, China)
Directed Input Generation (DIG) aims to automatically construct an input that reaches a specified target program location. Although recent studies have shown promising results for Large Language Models (LLMs) in function-level DIG, it remains unclear whether such effectiveness generalizes to realistic repository-level testing scenarios. Unlike function-level settings, repository-level DIG requires reasoning over long call chains, project-specific input semantics, and complex execution conditions distributed across real software repositories. To address this gap, we present a comprehensive empirical study of repository-level directed input generation with LLMs. Specifically, we construct a benchmark, RepoEval, containing hard-to-cover target lines across mainstream programming languages, and evaluate multiple state-of-the-art LLMs under different context and output settings. Our results show that repository-level DIG remains substantially more challenging than prior simplified settings. While advanced models can cover a portion of targets (from 16.02% to 37.35%), their effectiveness is strongly compromised by repository specific semantic constraints, context settings, output formats, and external execution conditions. Based on these findings, we design AgenticDIG, an agentic workflow that integrates target-oriented analysis, intention extraction, fault injection, and iterative feedback to improve repository-level DIG. Evaluation shows that AgenticDIG improves target coverage by 43.56% relative to the best baseline.

Article Search Article: ase26main-p2631-p doi:10.1145/3832783.3837537

A Comprehensive Evaluation of Code Language Models for Security Patch Detection
Nils Loose, Joseph Bienhüls, Kristoffer Hempel, Felix Mächtle, and Thomas Eisenbarth
(University of Lübeck, Germany)
Automated detection of vulnerability-fixing commits (VFCs) is critical for timely security patch deployment, as advisory databases lag patch releases by a median of 25 days and many fixes never receive advisories. Code language models are increasingly adopted for identifying , yet whether they can recognize a security fix from the code change itself remains unclear, as reported performance is shaped by commit messages, project-level data leakage, and uncertain data quality. We present a rigorous re-evaluation that jointly controls these factors through a unified framework consolidating 20 fragmented datasets spanning more than 180000 commits. Training 270 models from 125M to 80B parameters, we isolate the code signal under group-stratified code-only evaluation and assess the impact of model capacity and additional code context, ranging from intra-procedural enrichment to inter-procedural repository context. Model capacity yields clear but insufficient gains, and the evaluated context signals provide no reliable improvement under strict false positive budgets. At a false positive rate of 0.5

Article Search Article: ase26main-p2677-p doi:10.1145/3832783.3837538

Exact-IaC: Execution-Driven Exact Knowledge Injection for Reliable IaC Synthesis
Hua Ge and Minxue Pan
(Nanjing University, China)
The deployment and maintenance of software systems rely heavily on Infrastructure-as-Code (IaC) to translate high-level infrastructure intent into deployable configurations. While Large Language Models (LLMs) show potential in automating IaC synthesis, relying on closed-source cloud APIs poses severe data privacy risks. Conversely, deploying Small Language Models (SLMs) locally often encounters performance bottlenecks due to schema hallucinations and attention dilution in long-context scenarios. Furthermore, existing retrieval-augmented generation (RAG), multi-turn repair, and semantic topology parsing approaches struggle to resolve complex declarative resource dependencies.
In this paper, we propose Exact-IaC, an execution-driven framework designed to enhance SLMs for reliable IaC synthesis. It employs a divide-and-conquer strategy: it transitions from initial semantic intent approximation to an iterative repair loop that utilizes compiler feedback for deterministic schema injection. This approach effectively offloads the SLM’s parametric memorization burden.
Extensive experiments on the IaC-Eval benchmark demonstrate the effectiveness and generalizability of Exact-IaC across diverse open-weight SLMs, yielding 55.0% to 69.8% relative gains. Specifically, for our primary 30B backbone (Qwen3-Coder), the framework elevates the Pass@1 rate from 31.00% (the performance ceiling of evaluated baselines) to 49.34%, achieving a 59.2% relative improvement. Notably, this locally deployed SLM, augmented by Exact-IaC, outperforms a top-tier proprietary model (GPT-5.2-Codex) operating under its strongest baseline setting (48.47%). Furthermore, our trajectory-level error analysis reveals the L6 Paradox: even at extreme infrastructure complexity, SLMs fail predominantly due to micro-parameter hallucinations rather than macroscopic topological reasoning deficits. Ultimately, Exact-IaC establishes a zero-data-leakage orchestration pipeline, proving empirically that deterministic context precision can effectively substitute raw parameter scale for secure enterprise cloud provisioning.

Article Search Artifacts Available Article: ase26main-p2695-p doi:10.1145/3832783.3837539

TrackTest: Trajectory Generation and Prioritization for Testing Multiple Object Tracking System
Xinyu Gao, Haoxin Chen, Shuoxiao Zhang, Enyi Tang, Hengrui Xu, Haoxiang Tian, Minghui Wei, and An Guo
(Nanjing University, China; Nanyang Technological University, Singapore; Hong Kong Polytechnic University, China)
Multiple Object Tracking (MOT) is a fundamental component of the perception stack in Autonomous Driving Systems (ADS), providing essential state estimation for downstream decision-making. Despite the adoption of multi-sensor fusion to enhance reliability, MOT systems remain susceptible to tracking failures, which can lead to catastrophic safety incidents. Current perception testing and evaluation predominantly rely on manually annotated real-world datasets, which are behaviorally imbalanced, prohibitively expensive to scale, and insufficient to cover diverse dynamic interactions.
To bridge this gap, we propose TrackTest, an automated testing framework designed to systematically expose tracking failures in MOT systems. TrackTest transforms real-world seed scenarios into challenging test cases by synthesizing physically feasible and behaviorally diverse trajectories. We formulate six representative driving patterns as constraint-based optimization problems for behavior-controllable trajectory generation. To reduce computational overhead, we introduce a trajectory prioritization strategy that identifies high-risk scenarios most likely to trigger tracking failures. We evaluate TrackTest on six state-of-the-art MOT systems. Experimental results show that TrackTest effectively generate behavior-diverse trajectories and uncover diverse tracking failures.

Article Search Article: ase26main-p2727-p doi:10.1145/3832783.3837540

PELSE: Asymmetric-Evidence-Guided MCU Peripheral Model Synthesis
Guohao Wu, Chunlin Wang, Rui Yao, Feng Hong, Hongliang Liang, and Qiuping Yi
(Beijing University of Posts and Telecommunications, China; Fudan University, China; Beijing Institute of Computer Technology and Applications, China)
Faithful executable peripheral modeling remains a major bottleneck in microcontroller unit (MCU) firmware rehosting. Reference manuals provide broad but non-executable semantic descriptions, while binary analysis yields execution-grounded but incomplete observations. We present PELSE, a framework that synthesizes executable peripheral models by jointly leveraging these two asymmetric evidence sources. Our key insight is that manual-derived semantic structures and binary-derived execution witnesses can be unified under a shared register/field abstraction to mutually constrain model construction. Specifically, PELSE uses large language models (LLMs) to extract semantic candidates from vendor manuals and organize them into manual-derived finite-state machine (FSM) scaffolds, and performs memory-mapped I/O (MMIO)-aware symbolic execution to collect register/field-level witness fragments from firmware binaries. It then applies evidence-guided fusion in a unified register/field namespace to cross-validate and concretize compatible manual-derived transitions, recover well-formed trace-extracted transitions absent from the FSM scaffolds, isolate contradictions, and construct evidence-labeled fused FSMs before compiling them into QEMU-compatible peripheral models.
We evaluate PELSE using QEMU-referenced MMIO/IRQ conformance and an enhanced 106-test P2IM suite. PELSE achieves 97.0% conformance on a 42-test overlap set, improving over PERRY by 6.4 percentage points, and a 96.2% pass rate across all 106 tests, outperforming FUZZWARE by 5.6 percentage points. As downstream evidence of improved rehosting fidelity, PELSE-enabled rehosting supports security analyses that reproduce known vulnerabilities and uncover 15 previously unknown real-world vulnerabilities.

Article Search Artifacts Available Article: ase26main-p2735-p doi:10.1145/3832783.3837541

Benchmarking Automated Security Patch Backporting: How Far Are We?
Jincheng Yang, Yulong Fu, Chengwei Liu, Lyuye Zhang, Fangyuan Zhang, Bingyang Ren, Yang Liu, and Hui Li
(Xidian University, China; Nankai University, China; Nanyang Technological University, Singapore)
Automated security patch backporting is critical for mitigating N-day vulnerabilities. Recent tools have reported high success rates, often exceeding 80% on their respective datasets. However, these evaluations are typically confined to homogeneous environments, such as within the same repository or specific project versions. Consequently, it remains unclear how well these tools generalize beyond their originally targeted scenarios.
We present Porting Benchmark, a curated dataset of 1,234 security patch backporting cases spanning cross-version, cross-branch, and cross-repository scenarios, paired with a common evaluation framework. Using this benchmark, we evaluate five state-of-the-art tools spanning traditional program analysis, LLM-based prompting, and LLM agents under aligned evaluation settings. Our results show that aligned evaluation changes the apparent performance landscape: PortGPT and TSBPort remain comparatively strong on the Replication Dataset, while FixMorph and Mystique degrade substantially under the common protocol. Performance degrades sharply on structurally complex patches: the best commit-level success rate falls from 85.2% on Type-I patches to 24.0% on Type-IV. We identify four root-cause categories (missing target API awareness, cross-version semantic mismatch, non-local dependency propagation failure, and patch construction or localization failure) and derive concrete directions for next-generation tool design. On a 45-case dynamically validated subset with verified test cases and constructed POCs, we further observe that reference-based benchmark scores do not fully capture real-world remediation: exact match sharply under-credits harder target adaptations, while executable validation reveals residual integration failures in the target that static reference agreement misses. Executable-feedback refinement provides limited but measurable recovery on the hardest executable cases.

Article Search Article: ase26main-p2788-p doi:10.1145/3832783.3837542

When Knowledge Changes: Metamorphic Testing of RAG Systems with Mutations
Jinhan Kim, Samuele Pasini, and Paolo Tonella
(USI Lugano, Switzerland)
Retrieval-Augmented Generation (RAG)-based LLM systems rely on external document corpora that can evolve and change over time. However, current evaluation methodologies (e.g., RAGAS) assess correctness against static snapshots, failing to detect faults when routine updates, factual changes, or noise alter the underlying data. We introduce a metamorphic testing framework that evaluates the consistency of RAG systems under corpus evolution. We formalise a fault taxonomy and 11 mutation operators that systematically perturb the system at both the pre-chunk (retrieval index) and post-chunk (retrieved context) levels. An empirical evaluation across five datasets and over 28k mutants reveals metamorphic violation rates of 4.9-10.2%. In a meta-evaluation against ground truth, our metamorphic oracle achieves F1 scores of 0.927-1.000, while the best RAGAS metric reaches only 0.570. Finally, we provide actionable insights into mitigating these faults through retrieval re-configuration, generator upgrades, and LLM-based reranking.

Article Search Article: ase26main-p2811-p doi:10.1145/3832783.3837543

Testing Static Analyzers via Semantic-Preserving Mutators Learned from Real-World Refactoring Practice
Meilin Li, Kaixuan Li, Zifan Xie, Shiyu Qiu, Ming Wen, Maolin Sun, and Hongyu Zhang
(Chongqing University, China; Nanyang Technological University, Singapore; Huazhong University of Science and Technology, China; Nanjing University, China)
Static analyzers reason about program behavior without execution and report issues when code violates predefined rules. A key limitation is their production of inaccurate, incomplete results, generating spurious warnings and missing critical issues. Metamorphic testing is the most commonly used method to efficiently detect bugs in static analyzers. It requires designing a set of semantic-preserving mutators. When a mutator is applied to a seed program, any discrepancy in the analyzer's reports for the original and mutated versions indicates a potential bug. Thus, efficiently generating large numbers of semantic-preserving mutators is critical. We observe that during routine software maintenance, developers often restructure code to improve readability or optimize it without changing external program behavior. Our key insight is that mining and generalizing these patterns from real-world refactoring practice enables the automatic construction of numerous mutators that reflect the distribution of real-world code variants. We present SAFuzzer, a framework that mines such refactoring practice and distills these edits into semantic-preserving mutators. We ultimately generate 313 mutators and apply them to detect bugs in five widely used Java static analyzers. SAFuzzer detects 42 previously unknown bugs. Compared with five state-of-the-art baselines, SAFuzzer detects the most bugs, demonstrating the value of learning diverse mutators from real-world refactoring practice.

Article Search Article: ase26main-p2906-p doi:10.1145/3832783.3837544

ReSEDA: Automatic and Precise Program-Level ReDoS Triage via Interprocedural Taint Analysis and Agentic Exploit Generation
Rongchen Li, Weihao Su, Chengyao Peng, Hong Huang, Haiming Chen, and Guiyi He
(Institute of Software at Chinese Academy of Sciences, China; University of Chinese Academy of Sciences, China)
Regular expressions (regexes) are widely used in computer science, particularly in web services. However, Regular Expression Denial-of-Service (ReDoS) has become a growing security threat. ReDoS is an asymmetric Denial-of-Service attack that exploits the super-linear runtime behavior of regex engines. However, existing ReDoS detection tools only report whether regexes cause the super-linear matching behavior, instead of considering whether the behavior can be triggered in the program context. As a result, triaging such findings remains expensive for developers in practice.
We present ReSEDA, an automatic and precise workflow that triages ReDoS alerts and generates executable proofs of concept (PoCs) for vulnerable JavaScript programs. The workflow employs (i) Interprocedural Finite Distributive Subset (IFDS) taint analysis, (ii) Satisfiability Modulo Theories (SMT) solving for supported path-feasibility and ReDoS constraints, and (iii) Large Language Model (LLM) agents for PoC construction with Model Context Protocol (MCP) tools that provide static evidence.
From 86 most starred JavaScript projects in GitHub, we confirm that among 637 regexes reported to be vulnerable, there are only 63 regexes which can be exploited by injection through external interfaces, where ReSEDA is able to identify and generate PoCs for 61 of them automatically. We also discover that the assistance of ReSEDA reduces analyst-hours by 8.78× using $2.01 per vulnerability, or by 7.86× for free in a local-LLM setting. Furthermore, ReSEDA uncovered 27 impactful ReDoS vulnerabilities with assigned CVE IDs, including 8 disclosed fully automatically.

Article Search Article: ase26main-p2923-p doi:10.1145/3832783.3837545

A Few Pages of Markdown: Committed AI Configuration and Lower Quality Cost after Coding-Agent Adoption
Yegor Denisov-Blanch, Shyam Agarwal, Pavel Azaletskiy, Hao He, Rylan Schaeffer, Brando Miranda, Bogdan Vasilescu, and Sanmi Koyejo
(Stanford University, USA; Carnegie Mellon University, USA; Grid Dynamics, USA)
Coding agents increase development velocity but also technical debt. Prior work reports only average effects across adopters, hiding wide differences between teams. We introduce RAMP (Repository AI Maturity Profile), a four-level cumulative maturity model grounded in version-controlled artifacts that teams commit to configure AI tools. RAMP runs from behavioral rules and coding standards through named agent definitions to multi-agent orchestration, with observed practice concentrated in the first three levels. Across 441 repositories the levels behave as a cumulative scale, and independent human annotation reproduces RAMP’s repository-level labels on 97

Article Search Article: ase26main-p2970-p doi:10.1145/3832783.3837546

Perception, Reasoning, and Action: A ReAct-Based Vulnerability Detection Framework with Repository-Level Reasoning
Xin-Cheng Wen, Hanyue Luo, Cuiyun Gao, Hao Zhu, Yang Xiao, and Ge Li
(Harbin Institute of Technology, Shenzhen, China; Peking University, China; Institute of Information Engineering at Chinese Academy of Sciences, China; University of Chinese Academy of Sciences, China)
The exponential growth in software vulnerabilities has created an urgent demand for automated vulnerability detection solutions. Recent research has emphasized the importance of repository-level context for vulnerability detection and has leveraged large language models (LLMs) to address this challenge. Although these methods show state-of-the-art performance, they face the following challenges: (1) Insufficient perception of repository-level contextual information. Current approaches primarily rely on the given input itself or predefined knowledge bases. They cannot adequately account for inter-file dependencies, such as import relationships and class hierarchies, that may be critical to understanding specific vulnerabilities. (2) Relying on manually designed actions in the LLM reasoning process. Specifically, existing methods do not effectively integrate reasoning (i.e., predicting vulnerability existence by LLMs) with action (i.e., dynamically retrieving additional or reduced context by tool). This limitation can lead to factual hallucinations throughout the detection process, leading to limited performance.
To address these challenges, we propose to model the repository-level vulnerability detection task from a Reasoning and Action (i.e., ReAct) paradigm and incorporate human workflows into multi agent-based LLMs, termed ReAct-Vul, which consists of two key modules: (1) a repository-scale knowledge slicing, which comprises an intra-procedural slicing process, an inter-procedural slicing process, and a ReAct-based perception component for extracting the vulnerability-related context and knowledge at the repository level. (2) a synergistic reasoning-action-driven detection module, which interleaves a dual-perspective vulnerability analysis (Action phase) and detection-reflection loop (Reasoning phase). This synergistic approach enables the framework to explicitly consider both vulnerability presence and absence, generate reasoning explanations and refine subsequent targeted actions for vulnerability detection. We evaluate ReAct-Vul on the real-world C/C++ dataset PrimeVul and SVEN to demonstrate its effectiveness in vulnerability detection. Specifically, ReAct-Vul outperforms the best baseline by 18.4% on SVEN and 8.7% on PrimeVul in terms of accuracy. Furthermore, we demonstrate that the proposed ReAct paradigm enhances the reasoning capabilities of different models, achieving a 15.9% improvement in the accuracy of SVEN.

Article Search Article: ase26main-p3189-p doi:10.1145/3832783.3837547

TDiFf: Detecting Bugs in DataFrame Systems via Transferred DBMS Test Cases
Jiaxin Hu, Shaowei Chen, and Rongxin Wu
(Xiamen University, China; Fujian Agriculture and Forestry University, China)
DataFrame systems are core components of modern data analytics platforms and achieve high performance through various optimization techniques. However, these optimizations can introduce subtle defects that are difficult to detect, undermining system reliability. Existing testing approaches primarily rely on manually written unit tests or automated unit test generation tools, which often produce invalid or low-quality test cases and have limited ability to exercise complex API sequences and optimization-induced behaviors.
In this paper, we present TDiFf, a novel LLM-based testing framework for DataFrame systems. TDiFf leverages the database-like characteristics of DataFrame systems by translating SQL query statements into Python DataFrame test cases for differential testing. To address limitations of LLMs in mapping SQL constructs to DataFrame APIs and preserving semantic consistency, TDiFf incorporates feature knowledge extracted from SQL and DataFrame API documentation. In addition, it utilizes data flow graphs derived from SQL query plans to guide the generation of syntactically valid and semantically consistent DataFrame API sequences.
We evaluate TDiFf on six widely used DataFrame systems: Pandas, Dask, CuDF, Modin, PySpark, and Polars. The results show that more than 99% of the generated DataFrame test cases are valid. Using these test cases, TDiFf uncovers 35 unique and previously unknown bugs confirmed by developers, and outperforms state-of-the-art baseline approaches by 2.5× to 14×.

Article Search Article: ase26main-p3190-p doi:10.1145/3832783.3837548

MiniPoly: Automatic Extraction of Efficient JavaScript Polyfill from Language Specification
Jungwoong Kim, Youngmin Cho, and Jihyeok Park
(Korea University, Republic of Korea)
A polyfill is a JavaScript implementation of built-in APIs for older execution environments lacking native support. Because developers rely on them to run modern features everywhere, strict conformity to the official language specification (ECMA-262) is crucial. However, manual implementation remains the industry standard; this process is labor-intensive, inherently error-prone, and frequently leads to spec-miscompliance bugs in production.
To address this, we present MiniPoly, the first compilation framework to automatically extract correct and optimized polyfills directly from the ECMA-262 specification. To ensure correct generation, MiniPoly employs a static analysis that resolves the specification’s internal control-flow representations (i.e., completion records) and translates them into equivalent JavaScript constructs. Furthermore, it utilizes rewrite rules to apply user-defined optimization patterns, replacing algorithmic bottlenecks in the specification with highly efficient implementations. Evaluation shows that MiniPoly successfully generates polyfills for 89 built-in APIs in ECMAScript 2025 (ES2025) using 17 user-defined rewrite rules. It produces a more spec-compliant polyfill than production-standard libraries like core-js and es-shims with a competitive performance profile. Additionally, we demonstrate that MiniPoly can generate polyfills even for the next draft release of ECMA-262 and early-stage proposals, highlighting its potential to future-proof development.

Article Search Artifacts Available Article: ase26main-p3249-p doi:10.1145/3832783.3837549

IcFuzz: Fuzzing Isaac Sim with Semantic Stage Guidance and Multi-level Mutation
Zhixiang Chen, Zhuangbin Chen, Ruoxi Jia, Zeqin Liao, Wei Li, Jinyang Liu, and Zibin Zheng
(Sun Yat-sen University, China; Nanyang Technological University, Singapore; Chinese University of Hong Kong, Hong Kong)
Robotics simulators serve as a foundational infrastructure for embodied AI, facilitating safe and scalable robotic system development. NVIDIA Isaac Sim has emerged as one of the most popular simulators, distinguished by its GPU-accelerated physics engine and photorealistic rendering, which enable high-fidelity modeling of complex environments. However, its inherent complexity inevitably introduces software bugs that can compromise simulation reliability. Existing fuzzing approaches struggle to test Isaac Sim effectively due to challenges of context-aware object semantics, hierarchical simulation control, and a vast simulation state space.
In this paper, we propose IcFuzz, the first fuzzing approach for Isaac Sim. IcFuzz first performs an LLM-based semantic stage segmentation, decomposing simulation programs into structured stages that capture context-aware object semantics. Guided by this information, IcFuzz designs multi-level mutation operators to systematically exercise the simulator across hierarchical granularities. To efficiently navigate the vast simulation state space, IcFuzz employs a multi-armed bandit algorithm to adaptively schedule mutation operators. Experimental results show that IcFuzz outperforms the baselines in terms of both code coverage and bug detection. Specifically, IcFuzz achieves approximately 190%–205% of the code coverage of the baselines and detects an average of 3.7 unique crashes over three rounds of 12-hour tests, while no crashes are detected by the baselines. Moreover, IcFuzz has uncovered 11 bugs over approximately four months, 9 of which have been confirmed or fixed by the developers.

Article Search Article: ase26main-p3261-p doi:10.1145/3832783.3837550

How Well Do LLMs Generate Taxonomies in the SE Domain? A Multi-perspective Evaluation Framework
Sota Nakashima, Yuta Ishimoto, Masanari Kondo, Tao Xiao, and Yasutaka Kamei
(Kyushu University, Japan)
Taxonomies provide a shared conceptual framework for organizing heterogeneous observations in software engineering (SE) research. Manually constructing such taxonomies is labor-intensive and requires annotators with expertise in the SE domain. While advances in Large Language Models (LLMs) have led to the emergence of automated taxonomy generation methods outside the SE domain, their applicability to technically complex SE artifacts remains unclear. In this experience paper, we present the first comprehensive empirical evaluation of how state-of-the-art automated methods perform on SE artifacts through a multi-perspective evaluation framework, including taxonomy quality, alignment with taxonomies defined by human experts, reliability under independent annotation, and efficiency. To support this evaluation, we systematically collect seven SE papers with publicly available artifacts and human-defined taxonomies, and conduct experiments using two automated methods (TnT-LLM and CLIMB) with five state-of-the-art LLMs. Our evaluation reveals a clear trade-off: TnT-LLM constructs high-quality taxonomies comparable to human-defined ones but incurs substantially higher cost and runtime and tends to generate overly complex taxonomies, whereas CLIMB is 15–40× faster and 8–49× cheaper but tends to score lower on quality when technical inference beyond surface-level similarity is required. These findings suggest that TnT-LLM and CLIMB can be used in practical situations in the SE domain, while researchers should first assess the complexity of the generated taxonomies and their cost using a subset of the target data to decide whether to use automated methods or human experts. Our work represents a first step toward a systematic understanding of automated taxonomy generation in SE, offering actionable insights for future research and practice.

Article Search Article: ase26main-p3263-p doi:10.1145/3832783.3837551

Implicit, Yet Impactful: Understanding Hidden Dependencies in Java Projects
Lyuye Zhang, Chengwei Liu, Fangyuan Zhang, Yiran Zhang, Yuan Zhou, and Yang Liu
(Nanyang Technological University, Singapore; Nankai University, China; Zhejiang Sci-Tech University, China)
As software usage continues to expand, package managers automatically resolve dependencies to construct a dependency graph based on user-specified requirements. These explicitly declared dependencies, known as direct dependencies, receive significant attention in terms of maintainability and security. However, implicit dependencies, which are not explicitly defined by users but are still directly utilized or referenced in their project code due to oversight, remain largely unnoticed. Unlike ordinary transitive dependencies, which may remain unused and invisible to the root, implicit dependencies are actively used yet undeclared, leaving their versions outside the project’s direct control. This lack of awareness poses substantial challenges related to security and maintainability.
In this study, we present the first study to treat implicit dependencies as the focal phenomenon and quantitatively characterize their lifecycle consequences for the Maven ecosystem. We meticulously collected and built a large-scale dataset with 1,157 libraries with 19,812 versions from the Maven Central Repository and 972 modules from GitHub. Our findings reveal that 34.12% of the analyzed dataset contains implicit dependencies, with two primary causes identified as key contributors to the issue. Among these, 48% introduce breaking changes due to version drift, and 36 CVEs have vulnerable methods directly used by root projects; 30.28% of implicit dependencies are affected by known vulnerabilities under the version-range convention SCA tools use for declared dependencies. Finally, we identified and analyzed four major countermeasures, providing actionable insights and practical implications for addressing this overlooked issue for stakeholders within the OSS ecosystem.

Article Search Article: ase26main-p3524-p doi:10.1145/3832783.3837552

Piece by Piece: Automating Combination Interaction GUI Testing via Planning and Dual Memory
Zhe Liu, Oujin Wang, Junjie Wang, Chunyang Chen, Mengzhuo Chen, Boyu Wu, Yuekai Huang, and Qing Wang
(Institute of Software at Chinese Academy of Sciences, China; Tsinghua University, China; TU Munich, Germany; University of Chinese Academy of Sciences, Beijing, China)
Combination interactions require coordinated operations across multiple GUI widgets within a page to complete a functional task, which are common in modern mobile apps. However, due to widget dependencies, dynamic interface transitions, and context-sensitive execution, existing automated GUI testing approaches based on predefined models or state exploration struggle to reliably complete such interactions. LLM-based GUI testing methods improve semantic understanding but often generate inconsistent actions due to the lack of coordination modeling and interaction context. To address these challenges, this paper proposes COMBDroid, an automated GUI testing approach that enables reliable completion of combination interactions through structured planning and dual memory. COMBDroid first detects combination interaction patterns from GUI structure, then generates page-level interaction plans and grounds them into widget-level executable actions. Inspired by Sudoku solving, COMBDroid maintains a dual-memory mechanism, where context memory preserves execution states within a functionality and experience memory captures reusable interaction patterns. COMBDroid is designed as an integrable module that activates when combination interactions are detected and allows existing testing tools to resume exploration after completion. We evaluate COMBDroid on 100 combination interaction functionalities collected from 100 mobile apps, comparing it with 15 state-of-the-art GUI testing baselines. COMBDroid achieves 103% higher passing rate than the best baseline and improves activity coverage by 65%-100% when integrated with existing testing tools. Furthermore, GUI testing tools integrated with COMBDroid detect 37 new crash bugs in real-world apps, with 26 fixed and the remaining confirmed by developers, demonstrating its effectiveness in improving GUI testing.

Article Search Article: ase26main-p3565-p doi:10.1145/3832783.3837553

The CodeInverter Suite: Structure- and Data-Aware Binary Decompilation with Efficient LLMs
Peipei Liu, Jian Sun, Rongkang Sun, Li Chen, Zhaoteng Yan, Xiaoling Zhang, Dawei Wang, Dapeng Sun, Peizheng Zhang, and Dan Li
(Zhongguancun Laboratory, China; Tsinghua University, China)
Binary decompilation plays a vital role in various cybersecurity and software engineering tasks. Recently, end-to-end decompilation methods powered by large language models (LLMs) have attracted increasing attention for their ability to generate highly readable source code with minimal human intervention. However, existing LLM-based methods still struggle with reconstructing program structure and logic, achieving accurate data recovery, ensuring data security and privacy, and maintaining computational efficiency.
To address these challenges, we propose the CodeInverter Suite, with three main pieces: (1) the CodeInverter Workflow (CIW) is a novel prompt engineering method that incorporates control flow graphs (CFG) and explicit data mappings to enhance structure reconstruction and data recovery during decompilation; (2) building upon CIW, we construct the CodeInverter Dataset (CID), a large-scale domain-specific dataset containing 8.69 million samples enriched with CFGs and data mapping information; (3) we develop CodeInverter Models (CIMs), two lightweight LLMs with 1.3B and 6.7B parameters, enabling efficient inference in privacy-sensitive and resource-constrained environments.
Extensive experiments on two benchmark datasets demonstrate that CIW significantly enhances the decompilation performance of various LLMs, with average improvements of 13.07% in re-executability and 23.94% in re-compilability. For our proposed decompilation model, CIM-6.7B achieves state-of-the-art performance in terms of re-executability and readability, outperforming existing LLMs—even with over 100× more parameters—by an average of 11.03% and 6.27%, respectively.

Article Search Article: ase26main-p3581-p doi:10.1145/3832783.3837554

Evaluating Inference-Time Defenses against Package Hallucination in LLM-Generated Code
Alberick Euraste Djire, Iyiola E. Olatunji, Melissa Tessa, Earl T. Barr, Jacques Klein, and Tegawendé F. Bissyandé
(University of Luxembourg, Luxembourg; University College London, UK)
LLMs are increasingly used for code generation, yet they frequently hallucinate non-existent software packages, creating exploitable entry points into the software supply chain. We make four contributions to this problem. First, we show that prior evaluation methodologies systematically inflate hallucination rates by misclassifying standard-library modules as hallucinations in some languages. For Python, the overestimation reaches 9.4 percentage points. Second, we evaluate seven inference-time defenses for mitigating package hallucinations, including five guided decoding strategies (Greedy, Contrastive, DoLa, Nudging, and Active Layer-Contrastive Decoding), an iterative self-refinement approach (Self-Refine), and a Retrieval-Augmented Generation (RAG)-based defense. Across eight models spanning five families and four programming languages (Python, JavaScript, Ruby, Rust), RAG reduces the package hallucination rate (PHR) in 18 of 32 model–language configurations. Third, we introduce Package Utility (PU) to assess whether defenses preserve valid and task-relevant recommendations. Among strategies evaluated, Greedy decoding provides the strongest average mitigation–utility trade-off. Fourth, we stress-test all strategies under adversarial prompts seeded with fabricated package names and find that PHR surges by up to 45 percentage points relative to standard prompts, with Ruby consistently the most vulnerable language (80.9–95.2%). Under adversarial conditions, RAG and Self-Refine outperform all decoding-only strategies, indicating that robust defense requires either external grounding or iterative self-verification when prompts are actively hostile.
Our results recast package hallucination as both a measurement problem and a decoding-time control problem, and they demonstrate that the choice of defense must be matched to the threat model and recommendation utility.

Article Search Article: ase26main-p3592-p doi:10.1145/3832783.3837555

Post-hoc Attention Steering of Large Language Models for Robust Code Understanding under Obfuscation
Xiaokai Rong, Aashish Yadavally, and Tien N. Nguyen
(University of Texas at Dallas, USA; University of Central Florida, USA)
Code obfuscation is widely used in software systems and malware to conceal program logic and hinder analysis, posing significant challenges for both human developers and automated tools. While large language models (LLMs) have shown strong capabilities in code understanding, our preliminary study shows that LLM performance significantly degrades on obfuscated code, suggesting a reliance on superficial lexical cues rather than deep semantic reasoning. To address this limitation, we propose CodeSteer, a novel attention steering approach that reallocates model attention toward semantically relevant program elements, including backward slices for output prediction and control-flow paths for execution reasoning. Our method integrates lightweight program analysis with inference-time attention steering to guide LLMs toward the core input-to-output dependencies of a program. Experiments across multiple models and datasets demonstrate that CodeSteer significantly improves performance on obfuscated code, often recovering comparable accuracy to the level of unobfuscated programs. We also show CodeSteer’s practical utility through a case study on buffer overflow detection, highlighting its potential for malware/vulnerability analysis and reverse engineering of obfuscated code.

Article Search Article: ase26main-p3738-p doi:10.1145/3832783.3837556

PAIChecker: Uncovering and Checking PR-Issue Misalignment in SWE-Bench-Like Benchmarks
Manyi Wang, Junjielong Xu, and Pinjia He
(Chinese University of Hong Kong, Shenzhen, China)
SWE-bench-like benchmarks are widely used for evaluating LLM's issue resolution capability. They typically follow a common construction pipeline: each PR (Pull Request) is paired with its linked issue by extracting issue references from the PR description; the issue description is used as the problem statement, and the PR patch serves as the test oracle. However, due to the inherent complexity of developing and maintaining large repositories, such PR-Issue pairings are often misaligned in practice. In this work, we systematically study SWE-bench Verified instances, finding that 13.6% exhibit misalignment across five patterns in eleven fine-grained scenarios. To enable reliable and scalable construction of those benchmarks in the future, we propose PAIChecker, a multi-agent system for checking PR-Issue misalignment in SWE-bench-like benchmarks. Specifically, PAIChecker adopts a three-phase design that combines specific pattern identification, cross-agent label synthesis, and code-level validation, thereby enabling more accurate, generalizable, and progressively verified detection. Experiments on SWE-Gym and SWE-bench Multilingual show that PAIchecker achieves the best performance across all four LLM backbones, reaching up to 92.12% and 91.67% binary accuracy, respectively.

Article Search Article: ase26main-p3982-p doi:10.1145/3832783.3837557

Can Large Language Models Decompose User Stories into Tasks? Exploring the Role of Prompting Strategies and Models
Delina Ly, Fatma Başak Aydemir, and Fabiano Dalpiaz
(VX Company, Netherlands; Utrecht University, Netherlands)
Refining user stories (USs) into tasks is an important step in agile software development, as it translates software requirements into project management artifacts that are assigned to team members. This story-to-task decomposition can be tedious. In this paper, we propose and evaluate DELTA, a Large Language Model (LLM)-based approach that automatically generates tasks from USs and assesses the quality of the resulting decompositions. We conduct a two-phase empirical study to evaluate DELTA. In the first phase, we identify the best-performing DELTA configuration by comparing different prompting strategies, LLM families, and sizes using text similarity metrics on human-created and LLM-generated decompositions. The comparison based on these metrics did not identify a single consistently best-performing configuration. Therefore, in the second phase, we conducted an expert evaluation to determine whether the observed differences in text similarity are reflected in expert judgments and whether DELTA’s criteria-based evaluation module, which uses LLM judges, aligns with expert judgments. Our results show that DELTA’s evaluation module produces rankings that align more closely with expert judgments than text similarity metrics. Moreover, experts generally prefer the top three LLM-generated decompositions, as ranked by LLM judges, over those created by humans, and they express a willingness to adopt DELTA.

Article Search Artifacts Available Article: ase26main-p4215-p doi:10.1145/3832783.3837558

Certified Program Synthesis with a Multi-modal Verifier
Yueyang Feng, Dipesh Kafle, Vladimir Gladshtein, Vitaly Kurin, George Pîrlea, Qiyuan Zhao, Peter Müller, and Ilya Sergey
(National University of Singapore, Singapore; Neapolis University Pafos, Cyprus; ETH Zurich, Switzerland)
Certified program synthesis (a.k.a. vericoding) is the process of automatically generating a program, its formal specification, and a machine-checkable proof of program/specification alignment from a natural-language task description. Two key challenges make vericoding difficult. First, specifications synthesised from natural language descriptions are often either too weak to be meaningful or too strong to be implementable, yet existing approaches lack systematic means to detect such defects. Second, the program verifiers used to validate results are fragmented: each tool supports a particular reasoning mode—auto-active (e.g., Dafny, Verus) or interactive (e.g., Rocq, Lean)—with its own trade-off between automation and expressivity. This forces each synthesis methodology to target a single paradigm, limiting the tasks it can handle. We overcome both challenges by structuring the synthesis workflow in stages around a multi-modal verifier—a single tool that combines dynamic validation, automated proofs, and interactive proof scripting within one foundational framework. We realise this idea in LeetProof, a new agentic pipeline built on Velvet, a multi-modal program verifier built in the Lean theorem prover. Multi-modality enables LeetProof to validate generated specifications via randomised property-based testing before any code is synthesised, decompose the synthesis task into sub-problems guided by verification conditions, and delegate residual proof obligations to frontier AI provers specialised for Lean. We evaluate LeetProof on an extensive benchmark suite derived from prior work on certified synthesis. Our specification validation uncovers defects in existing reference benchmarks, and LeetProof's staged pipeline achieves a significantly higher rate of fully certified solutions than a single-mode baseline at the same fixed budget—consistently across two different frontier LLM backends.

Article Search Artifacts Available Article: ase26main-p4427-p doi:10.1145/3832783.3837559

Graphectory Viewer: A Tool for Process-Centric Analysis of Agentic Software Trajectories
Charlie Jyu, Shuyang Liu, and Reyhaneh Jabbarvand
(University of Illinois at Urbana-Champaign, USA)
We present Graphectory Viewer, a web-based tool for interactive, process-centric analysis of software-agent trajectories. Building on the Graphectory representation introduced in our previous work, Graphectory Viewer transforms heterogeneous raw trajectories into phase-aware graphs that connect low-level execution details with higher-level behavioral structures. The tool supports trajectories from multiple agent frameworks and provides interactive graph construction; node-level inspection of thoughts, actions, and observations; search and filtering over large trajectory collections; and Sankey-style summaries of problem-solving phase transitions. These capabilities enable researchers and practitioners to inspect individual executions, identify recurring behavioral patterns, compare successful and failed runs, and analyze large trajectory corpora beyond final task outcomes. To support reproducibility and further research, we release Graphectory Viewer as an open-source artifact together with documentation, precomputed graphs, and the large-scale trajectory corpus.

Article Search Article: ase26tool-p2-p doi:10.1145/3832783.3834593

VidSumEval: A Web Platform and Benchmark for Evaluating AI-Generated Programming Video Summaries
Osama AlGhamdi, Ameer AlGhamdi, Faisal Alorabi, Mohammad D. Alahmadi, Abdulmalik Alzahrani, and Abdullah Almazroui
(University of Jeddah, Saudi Arabia)
Programming tutorial videos on YouTube are a widely used learning resource, but their length often makes content review inefficient. AI-based summarization tools can produce shorter versions of these videos, yet existing evaluation relies almost exclusively on similarity-based metrics that measure surface overlap rather than whether the summaries actually support learning. We present VidSumEval, a web platform and benchmark for evaluating AI-generated programming tutorial summaries from a learning-centered perspective. The benchmark pairs 10 tutorials in C, Java, and Python with summaries from two commercial engines (VEED and NotebookLM) and auto-generated comprehension quizzes; users can also submit new YouTube tutorials for offline review. We validate the benchmark with a within-subject study (n=20) measuring both quiz-based comprehension and Likert-scale ratings on completeness, clarity, and coherence; learning effectiveness is reported as the per-participant difference between summarized- and original-video quiz scores. The platform is available at https://progsumeval.onrender.com; the source code and dataset are archived at https://doi.org/10.5281/zenodo.21763079; and the screencast is available at https://youtu.be/iBB-5HfRVSk.

Article Search Artifacts Available Article: ase26tool-p3-p doi:10.1145/3832783.3834594

BESSER-NN: Visual Neural Network Design and Automated Code Generation with the BESSER Low-Code Platform
Nadia Daoudi, Armen Sulejmani, and Jordi Cabot
(Luxembourg Institute of Science and Technology, Luxembourg; University of Luxembourg, Luxembourg)
Neural networks (NNs) are a fundamental component of AI-enhanced software systems, often built following model-driven engineering (MDE) / low-code approaches. Yet, while MDE tools support the modelling of most system components, designing and implementing NNs remains a manual, repetitive, and error-prone task that requires expertise in framework-specific APIs such as PyTorch or TensorFlow. In this paper, we address this gap by extending the BESSER web modelling editor to support NN diagram design and automatic code generation. Our tool relies on a formal NN metamodel, which provides a framework-agnostic representation of neural networks that is automatically converted to runnable PyTorch and TensorFlow code, covering both architecture definition and training and test processes. We validate our tool on a set of representative NN architectures and demonstrate correct code generation across both frameworks. By building on the BESSER platform, our work paves the way for treating NNs as modelled artifacts within a unified MDE workflow. Our tool is open-source, released under the MIT license, and accessible at https://editor.besser-pearl.org/. A video demonstration is available at https://youtu.be/DCb6qwnnlQc

Article Search Article: ase26tool-p5-p doi:10.1145/3832783.3834595

TraceGate: A Framework for Policy-Controlled Failure Evidence in LLM-Assisted Program Repair
Nicolas Schuler, Vincenzo Scotti, and Raffaela Mirandola
(KIT, Germany)
TraceGate is a reusable research artifact for policy-controlled disclosure of crash evidence in LLM-assisted program repair. Rather than relying on fixed evidence bundles or unconstrained LLM-driven tool usage, TraceGate mediates the repair process by capturing structured crash snapshots, selecting evidence and budget actions according to configurable policies, materializing the selected evidence for existing repair backends, validating candidate patches, and logging decisions and outcomes. The artifact provides backend adapters, benchmark workflows, evaluation scripts, and recorded logs to enable reproducible studies of failure-time observability and cost-success trade-offs. In evaluations across four Python benchmarks, five repair backends, and three model sizes, TraceGate recovered an average of 20.5% of residual failures, i.e., tasks that remained unsolved after the underlying backends had exhausted their own repair attempts.
Screencast: https://youtu.be/Gcr6A7tVBis Artifact: https://github.com/NicolasSchuler/tracegate Data: https://zenodo.org/records/19678467

Article Search Artifacts Available Article: ase26tool-p6-p doi:10.1145/3832783.3834596

SAF: An AI-Agent-Ready and Browser-Accessible Static Analysis Framework for LLVM IR
Yuekang Li, Wei Li, Wei Song, Yi Liu, and Gelei Deng
(University of New South Wales, Australia; Griffith University, Australia; Nanyang Technological University, Singapore)
Static program analysis finds bugs before software ships, and modern checkers for C/C++ depend on whole-program infrastructure: pointer analysis, value-flow tracking, taint analysis, and IFDS-based data flow on top of LLVM. Today’s LLVM-IR frameworks fall short. SVF, Phasar, and Lotus expose their analyses through C++ APIs that require subclassing, recompilation, and pinned LLVM builds; onboarding takes months. CodeQL replaces the C++ API with a domain-specific query language. None is designed for the AI coding agents that increasingly drive software-engineering work. We present the Static Analyzer Factory (SAF), an LLVM-IR analysis framework that combines a Rust kernel exposed to Python through PyO3 zero-copy bindings, a WebAssembly and Pyodide browser playground that runs the same SDK with no install, a declarative YAML language for function specifications that the built-in checkers consume at runtime, and two shipped coding-agent skills installable in Claude Code and Codex. As an illustrative case, SAF’s bundled memory-safety checkers were themselves authored end-to-end through this workflow using Claude Code on the Opus 4.6 model. On the NIST Juliet C/C++ benchmark, these checkers achieve the best F1 on three of four memory-safety CWEs, each measured against whichever of SVF and Lotus ships a corresponding checker. Code: https://github.com/Static-Analyzer-Factory/static-analyzer-factory. Documentation and browser playground: https://static-analyzer-factory.github.io/static-analyzer-factory/. Demo video: https://youtu.be/Cj58VvNCrlw.

Article Search Article: ase26tool-p8-p doi:10.1145/3832783.3834597

TLCrafter: Peeling the Layers of Natural Language to Reveal Temporal Logic Specifications
Cheng Wen, Wenjun Ke, Xiao Liang, Zhi Ma, Xiaoyong Xue, Shengchao Qin, and Cong Tian
(Xidian University, China; Northwest University, China; Peking University, China)
Natural-language (NL) requirements are widely used in industrial systems, but formal verification typically requires precise temporal logic (TL) specifications. This paper presents TLCrafter, an interactive tool for translating NL requirements into TL formulas, including LTL, CTL, and STL. Given an NL requirement, TLCrafter constructs OnionL+, a hierarchical intermediate representation that supports more robust NL-to-TL formalization by combining LLM-based semantic decomposition with deterministic rule-based synthesis. Its visual interface allows engineers to inspect and correct the intermediate structure before formula generation. Experiments on domain datasets across LTL, CTL, and STL show that TLCrafter improves formula accuracy while maintaining full syntactic validity. A video of TLCrafter is available at https://youtu.be/jp-Pp_amjks. The tool and artifacts are available at https://sites.google.com/view/tlcrafter/.

Article Search Article: ase26tool-p13-p doi:10.1145/3832783.3834598

VidTutorAssistant: Automating Responses to Programming Tutorial Questions
Ahmad Tayeb, Sonia Haiduc, and Mohammad D. Alahmadi
(King Abdulaziz University, Saudi Arabia; Florida State University, USA; University of Jeddah, Saudi Arabia)
Programming tutorial videos on YouTube are an important information resource for software developers and students, and their comment sections have evolved into active spaces where viewers ask follow-up questions. The volume of these questions, however, often exceeds what content creators can address, leaving learners without the clarifications they need. We present VidTutorAssistant, a web platform that automates responses to viewer questions on programming video tutorials. VidTutorAssistant implements a retrieval-augmented generation pipeline that extracts a video’s transcript, then segments it and embeds it. It then classifies each viewer comment as being a question or non-question, retrieves the most relevant transcript segments to each identified question via cosine similarity, and then generates an answer to the question using an LLM (GPT-4), while grounding the response using the retrieved transcript segments as context. We validate VidTutorAssistant through a study on a subset of 440 user comments selected from a larger dataset of 105,553 comments extracted from 7,522 Python and Java tutorials. VidTutorAssistant is evaluated on various criteria: a) its ability to identify the programming language in a video, achieving a 0.99 accuracy; b) its ability to classify comments into questions and non-questions, reaching a 0.96 accuracy; and c) its ability to produce correct and complete answers to questions, producing 98% correct and 99.5% complete responses, compared with 89% and 90% for the original creators’ answers.

Article Search Article: ase26tool-p17-p doi:10.1145/3832783.3834599

ProofPulse: Interactive Proof Coverage Analysis for Dafny
Álvaro F. Silva, Ruben Martins, and Alexandra Mendes
(INESC TEC, Portugal; Universidade do Porto, Portugal; Carnegie Mellon University, USA)
Deductive verification ensures that an implementation satisfies its specification, but successful verification does not guarantee the quality of the specification. As such, weak specifications and redundant invariants may create overconfidence in "verified" code.
We present ProofPulse, a tool for Dafny that diagnoses specification quality using a three-valued proof coverage model. By analyzing proof dependencies, ProofPulse distinguishes between elements that contribute to specification intent, those used only for auxiliary checks, and those irrelevant to any proof obligation.
Evaluated against an oracle of 252 programs from the dafny-synthesis benchmark, ProofPulse provides a high-precision signal for specification weaknesses, particularly unnecessary preconditions and vacuous proofs. With unsat-core minimization, ProofPulse achieves perfect precision for precondition classification and reduces false positives across all evaluated categories. These results show that proof coverage is a practical complement to verification. Although it cannot fully capture semantic intent, it can reveal weak proof coupling in programs that otherwise appear fully verified.
Just as a pulse check distinguishes vitality from the mere absence of symptoms, ProofPulse exposes weaknesses in proofs that technically verify successfully but still suffer from inadequate or redundant code and specifications.
Demo: https://www.youtube.com/watch?v=8pO3NAodjoQ Code: https://github.com/VeriFixer/ProofPulse Prebuilt Docker image: https://doi.org/10.5281/zenodo.21174686

Article Search Artifacts Available Article: ase26tool-p18-p doi:10.1145/3832783.3834600

DT-Drive: A Tool for Deterministic Replay-Based Testing and Debugging of Autonomous Driving Systems
Sanjeetha Pennada, Matthew Leach, Olek Osikowicz, and Donghwan Shin
(University of Sheffield, UK)
While simulation is essential for testing autonomous driving systems (ADS), high-fidelity simulators like CARLA often exhibit non-determinism, leading to flaky simulation results. We present DT-Drive, a record-modify-replay framework that ensures deterministic ADS evaluation by decoupling the ego vehicle from the recorded driving scenario. This approach enables “ego-injection” for fair multi-ADS comparisons and “counterfactual replay” via environmental modifications (e.g., weather and traffic), facilitating targeted testing and debugging. Validated on 128 CARLA benchmark scenarios, DT-Drive achieved 100% deterministic evaluation results, effectively eliminating simulator-induced flakiness. We also demonstrate its utility for testing and debugging by conducting a controlled comparison of the TransFuser and TransFuser++ agents under identical and modified environmental conditions.
DT-Drive and the sample data are available at https://github.com/SanjeethaPennada/DT-Drive []. The tool demo video is available at https://youtu.be/HB1DIZSF5E0.

Article Search Article: ase26tool-p26-p doi:10.1145/3832783.3834601

FairLint-DL: An IDE-Native Tool for Fairness Debugging of Deep Learning Software
Archit Rathod and Saeid Tizpaz-Niari
(University of Illinois at Chicago, USA)
Existing fairness analysis tools predominantly operate as post-training evaluation frameworks, requiring practitioners to complete the full model development lifecycle before assessing bias. We present FairLint-DL, a Visual Studio Code extension that implements a shift-left approach to fairness testing by enabling pre-training, IDE-native bias detection directly on tabular datasets. FairLint-DL trains a configurable deep neural network as a proxy model and applies information-theoretic Quantitative Individual Discrimination (QID) metrics. Grounded in Shannon and min-entropy, QID quantifies the causal influence of protected attributes on predictions. The system implements a two-phase gradient-guided search algorithm for discovering discriminatory instances, a causal de- bugging pipeline that localizes bias to specific network layers and neurons via sensitivity analysis, and dual explainability engines using SHAP and LIME for feature-level attribution. Evaluation on three tabular benchmarks (Adult Census Income, German Credit, and Bank Marketing) reveals fairness concerns that vary widely across datasets: on Adult, 96.0% of analyzed instances exhibit QID above the 0.1-bit significance threshold, with a mean QID of 0.619 bits and a disparate impact ratio of 0.581, violating the four-fifths legal rule. FairLint-DL produces these results within 12 seconds on cached models, demonstrating the feasibility of integrating fairness analysis into the developer workflow without significant overhead.

Article Search Artifacts Available Article: ase26tool-p28-p doi:10.1145/3832783.3834602

VerusSeek: Retrieval-Augmented LLM-Based Proof Synthesis for Rust Programs
Yuchen Zhang, Cheng Wen, Jialun Cao, Dugang Liu, Zhiwu Xu, Yuwei Liu, and Shengchao Qin
(Xidian University, China; Hong Kong University of Science and Technology, Hong Kong; Shenzhen University, China; Institute of Software at Chinese Academy of Sciences, China)
Formal verification of Rust programs with Verus provides strong correctness guarantees, but developing the auxiliary specifications and proofs still requires considerable manual effort. Existing LLM-based proof-synthesis agents can automate part of this process, yet their effectiveness on non-trivial tasks is often limited by the lack of reusable proof patterns that are directly relevant to verification. This paper presents VerusSeek, a retrieval-augmented tool for LLM-assisted Verus proof synthesis for Rust programs. Instead of retrieving entire files or functions, VerusSeek indexes previously verified Verus code as semantic proof constructs, including contracts, assertions, loop invariants, lemmas, and proof blocks. For each verification task, the tool performs type-aware retrieval to select constructs that match the current proof obligation, expands them with bounded hierarchical context, and synthesizes lightweight structural loop-invariant hints for index bounds, frame conditions, and termination arguments. Verus checks each candidate proof, and the resulting diagnostics guide the next repair attempt. On 150 tasks from VerusBench, VerusSeek verifies 122 tasks, compared with 69 for AutoVerus and 85 for RagVerus. A demonstration video of VerusSeek is available at https://youtu.be/p2mb61gj95A.

Article Search Article: ase26tool-p29-p doi:10.1145/3832783.3834603

xDECAF: An Extensible Data Flow Diagram Analysis Framework for Information Security
Benjamin Arp, Felix Schwickerath, Alexander Vogt, Tom Hüller, Nils Niehues, and Nicolas Boltz
(KIT, Germany)
xDECAF is an extensible tool for architecture-based data flow analysis with a focus on information security. It combines an extended data flow diagram metamodel of labeled flows and nodes, a domain-specific constraint language with different flow operations, and a browser-based editor backed by an analysis engine. In this paper, we present the xDECAF tool library and a curated catalog of over 20 example models with documented constraints and expected violations, intended as a reusable dataset for the community. The tool has already been adopted by several research lines, providing concrete evidence of its utility. The tool, dataset, and a hosted online editor are publicly available.
Screencast: https://youtu.be/L-PGdWoPtlw Code & Dataset: https://github.com/DataFlowAnalysis • https://doi.org/10.5281/zenodo.20083877

Article Search Article: ase26tool-p34-p doi:10.1145/3832783.3834604

AISmellBench: A Reusable Benchmark of AI Code Smells in ML and LLM-Based Systems
Brahim Mahmoudi, Naouel Moha, Quentin Stiévenart, and Florent Avellaneda
(École de Technologie Supérieure de Montréal, Canada; Université du Québec à Montréal, Canada)
Artificial Intelligence (AI) systems are increasingly implemented through Machine Learning (ML) pipelines and Large Language Model (LLM) integrations, where design and implementation choices are scattered across data preprocessing code, model configuration, training scripts, provider APIs, and prompt/output logic. Poorly designed choices may form AI code smells, namely recurring implementation patterns that can undermine reproducibility, robustness, maintainability, and trustworthy behavior. Yet, evaluating AI code smell detection approaches remains difficult due to the lack of reusable benchmarks with validated ground truth. We present AISmellBench, a benchmark for AI code smells built through a reproducible pipeline combining GitHub repository mining, eligibility filtering, tool-assisted identification of potential code smell instances, and manual annotation with adjudicated validation artifacts. AISmellBench brings together 1,221 AI-based Python repositories, including 888 ML repositories and 333 LLM-based repositories. The released benchmark contains 1,265 annotated Python files with 3,714 validated AI code smell instances across 31 AI code smell types.
Demonstration video: https://youtu.be/xepUUGSHaiU Dataset availability: https://doi.org/10.5281/zenodo.20085308

Article Search Artifacts Available Article: ase26tool-p35-p doi:10.1145/3832783.3834605

CppPerf: An Automated Pipeline and Dataset for Performance-Improving C++ Commits
Tommy Ho, Khashayar Etemadi, and Zhendong Su
(ETH Zurich, Switzerland)
Recent progress in automated repair of performance bugs demands realistic, executable benchmarks. However, existing C++ performance benchmarks are largely built from competitive programming submissions, and recent real-world benchmarks predominantly target Python and .NET. To fill this gap, we present CppPerf-Mine, a configurable pipeline that mines execution-time-improving patches from open-source C++ repositories on GitHub by combining structural commit filtering, an LLM-based commit classifier, and a containerized build & test stage that produces fully reproducible Docker images for each patch. Using CppPerf-Mine, we build CppPerf-DB, a benchmark comprising 347 manually verified patches from 42 mature C++ repositories, 39% of which are multi-file, enabling the evaluation of repository-level repair tools. In our preliminary study, OpenHands correctly fixes only 13.5% of the patches in CppPerf-DB, confirming that real-world C++ performance repair remains an open challenge. CppPerf-Mine and CppPerf-DB are open-source and publicly available at: https://doi.org/10.5281/zenodo.20097425. In addition, a demonstration video is available at: https://www.youtube.com/watch?v=nixlupIgSdM.

Article Search Article: ase26tool-p37-p doi:10.1145/3832783.3834606

FlightRecorder: Tracing Fine-Grained Development with AI Assistants
Tobias Dick, Benedict Bliem, Kallistos Weis, Alisa Welter, and Sven Apel
(Saarland University, Germany)
AI coding assistants like GitHub Copilot are transforming software development. Yet fundamental questions remain about how code evolves as developers collaborate with AI agents, particularly at fine-grained levels of interaction -- a gap that persists partly because existing telemetry cannot distinguish agent-generated changes from inline completions and human edits, nor precisely track when and where they occur. To address this gap, we present FlightRecorder, a lightweight VSCode extension that collects fine-grained code evolution data without disrupting the workflows of developers. Our extension integrates three data sources: snapshots of agent-mode edits, inline completions, and chat history, all timestamped for precise attribution. Researchers can export these data in structured formats alongside anonymization and basic visualization functionality. By making such fine-grained evolution data accessible, FlightRecorder enables researchers to investigate fundamental questions about AI-assisted development. We provide a replication package, including the code for FlightRecorder and the data used throughout our paper, at https://doi.org/10.5281/zenodo.20134365. We further provide a screencast demo of our tool at https://youtu.be/bZR6aPROVMU.

Article Search Artifacts Available Article: ase26tool-p38-p doi:10.1145/3832783.3834607

ThreatCraft: Automated Attack Scenario Generation via Hybrid Rule-Based and LLM-Driven Reasoning
Dohee Kang, Jiwon Kwak, Geunwoo Baek, and Seungjoo Kim
(Korea University, Republic of Korea)
Recently, the growing complexity of modern Information Technology (IT) systems has made it increasingly difficult to identify realistic attack scenarios. Although various automated approaches have been proposed, challenges such as dependence on expert knowledge and hallucinations in LLM-based methods remain unresolved. We develop ThreatCraft, an attack-scenario generation tool that combines a rule-based engine with Large Language Models (LLMs). ThreatCraft constructs system-level attack paths using a Unified Kill Chain (UKC) and leverages them to guide LLMs in generating detailed attack scenarios. Experimental results across automotive, Industrial Control Systems (ICS), and enterprise domains show that ThreatCraft achieves an average F1-score of 0.984 and a semantic similarity score of 0.893, improving attack-path reconstruction accuracy and semantic similarity by up to 8.3% and 10.7%, respectively, compared with other tools. These results demonstrate that ThreatCraft can consistently generate realistic and structurally coherent attack scenarios across various domains. GitHub:https://github.com/HackProof/2026_ASE_ThreatCraft.git Demo:https://youtu.be/nrIHEKDLp2E

Article Search Artifacts Available Article: ase26tool-p39-p doi:10.1145/3832783.3834608

QUTest: A Native Testing Framework for Quantum Programs
José Campos
(University of Porto, Porto, Portugal; University of Lisbon, Lisbon, Portugal)
Quantum programs are often shared as OpenQASM 3 circuits, but tests are still written in host languages such as Python with Qiskit. We present QUTest, a native framework in which both programs and tests are standard .qasm files. Tests follow the Arrange / Act / Assert pattern, while configuration, runtime requirements, and assertions are encoded as pragma comments (//%), preserving compatibility with existing OpenQASM tools. QUTest provides 12 assertion types spanning deterministic, statistical, quantum-state, and structural checks, plus a linter and an environment-aware mode for running the same test across selected runtime versions in isolated environments. Its CLI supports automatic test discovery, runtime compatibility checks, and XML reports for continuous integration. We describe the pragma language, implementation, and a planned evaluation using coverage and mutation testing.

Article Search Artifacts Available Article: ase26tool-p42-p doi:10.1145/3832783.3834609