ISSTA 2026
Proceedings of the ACM on Software Engineering, Volume 3, Number ISSTA

Title Page

Article: issta26foreword-fm000-p (type: Frontmatter) doi:

Sponsors

Article: issta26foreword-fm003-p (type: Frontmatter) doi:

Editorial Message
Marcel Böhme and Cindy Rubio-González
(MPI for Security and Privacy, Germany; University of California at Davis, USA)

Article Search Article: issta26editorial-fm001-p (type: Editorial) doi:10.1145/3833374

Testing Computation Pushdown in Distributed Database Systems
Jinsheng Ba, Zuming Jiang, and Zhendong Su
(ETH Zurich, Switzerland; University of Hong Kong, Hong Kong)
Computation pushdown is a critical technique in distributed database management systems (DBMSs), enabling certain operations to be executed closer to the data to reduce network overhead and improve performance. However, its behavior depends on multiple factors beyond the input query itself, such as data distribution and resource utilization. This makes it difficult to validate correctness using only input queries in a black-box manner. Existing testing methods that rely solely on query manipulation cannot effectively control or predict pushdown behavior, and are therefore insufficient. In this paper, we introduce Controlled Pushdown Execution (CPE), a white-box method that enables systematic validation of computation pushdown. CPE modifies the source code of DBMSs to forbid a specific pushdown operator and compares the results. Any discrepancy reveals a bug. Our study shows that CPE can control all supported operators across different systems. We applied CPE to three production-grade distributed DBMSs: CockroachDB, TiDB, and YugabyteDB. CPE found 25 previously unknown and unique bugs, 14 of which are logic bugs---incorrect results. CPE finds 3x more bugs than historical bugs and can reproduce all historical bugs. Beyond computation pushdown, the core insight of controllable execution can generalize to other contexts (e.g., transaction schedule), providing a systematic way to uncover subtle logic bugs.
Article Search Article: issta26main-p12-p (type: Full Paper) doi:10.1145/3832092

Do Coverage and Mutation Scores of LLM-Generated Test Suites Correlate with Their Effectiveness? (Replicability Study)
Junda Zhao, Shurui Zhou, and Eldan Cohen
(University of Toronto, Canada)
Recent advances in large language models (LLMs) have driven growing interest in using LLMs to automate test generation. Prior work commonly evaluates generated test suites using proxy metrics such as code coverage and mutation score. However, studies by Inozemtseva et al. and Papadakis et al. show that, for human-written tests, correlations among coverage, mutation, and real-bug detection can largely vanish once test suite size is controlled, raising concerns about the validity of evaluations based on proxy metrics. It also remains unclear whether these conclusions carry over to LLM-generated tests, given that prevailing LLM-based test-generation workflows differ substantially from traditional approaches.
In this paper, we conduct a large-scale replication study of these two prior works using a wide range of test suites generated by a diverse set of LLMs, and re-examine the relationships among coverage, mutation, and real-bug detection effectiveness. Our findings diverge substantially from prior results. We show that the usefulness of coverage and mutation is highly context-dependent: in regression-style settings where the code provided to the LLM can be reasonably assumed bug-free, these metrics can provide meaningful signals when comparing across models; in another common scenario where the code-under-test may already be buggy and the goal is to expose the bug within the code-under-test, they no longer serve as reliable indicators. We also find little evidence that test suite size is a dominant confounder for correlations among coverage, mutation, and real-bug detection for LLM-generated tests. Based on these findings, we discuss how to interpret results from prior studies and provide actionable guidance for evaluating LLM-based test generation.
Article Search Artifacts Available Article: issta26main-p17-p (type: Full Paper) doi:10.1145/3832093

Reproduction Package for Article 'Do Coverage and Mutation Scores of LLM-Generated Test Suites Correlate with Their Effectiveness? (Replicability Study)' (doi:10.5281/zenodo.21437945): Replication package for a large-scale conceptual replication of Inozemtseva & Holmes (ICSE 2014) and Papadakis et al. (ICSE 2018) for LLM-generated test suites: do coverage and mutation scores still predict real-bug detection, and is test-suite size still the dominant confounder? The study covers 11 LLMs (13 model ...

BashCoder-R1: Towards Robust and Explainable Bash Script Generation with Robustness-Aware Group Relative Policy Optimization
Lei Yu, Peng Wang, Jia Xu, Jingyuan Zhang, Xin Wang, Jiajia Ma, Li Yang, Changzhi Deng, Zenghua Wang, and Fengjun Zhang
(Institute of Software at Chinese Academy of Sciences, China; University of Chinese Academy of Sciences, China)
Bash scripts underpin system administration, DevOps automation, and Continuous Integration/Continuous Deployment (CI/CD), where code quality directly determines system stability and security. Yet when Large Language Models (LLMs) are tasked with generating such scripts, two problems compound each other: models produce code without any accompanying justification for their design choices, and this absence of scrutiny correlates with scripts that silently harbor robustness flaws, including mishandled edge cases, unchecked failure modes, and fragile assumptions about the execution environment. We present BashCoder-R1, a framework that tackles both problems jointly, treating explainability as a design goal rather than a byproduct of correctness. The training pipeline proceeds in three stages. Continual Pre-training (CPT) first adapts the base model to the syntactic conventions and idioms specific to Bash. We then curate thousands of expert-validated reasoning-and-code samples and use them for Long Chain-of-Thought Supervised Fine-Tuning (L-CoT SFT), training the model to reproduce the deliberative, risk-averse thinking pattern of an experienced system administrator before it writes any code. Finally, Robustness-Aware Group Relative Policy Optimization (R-GRPO) directly refines the generation policy against a weighted reward combining syntax correctness, robustness as verified by the static analyzer shellcheck, and adherence to the required reasoning format. We evaluate BashCoder-R1 on BashBench, a benchmark we construct from 952 real-world automation tasks (773 single-line commands and 179 multi-line scripts), against a wide range of state-of-the-art baselines. BashCoder-R1 attains SyntaxPass of 100.00% / 94.97%, RobustWarnRate of 4.01% / 16.47%, RobustPass of 95.99% / 79.33%, FuncRate of 93.01% / 93.85%, and FullRate of 90.04% / 73.18% on single-line / multi-line tasks respectively, a relative FullRate improvement of 37.82% and 20.18% over the strongest baseline, DeepSeek-V3.2 (Reasoning). Human evaluation across Functionality, Robustness, and Clarity further confirms that BashCoder-R1's reasoning chains are rated highest in quality among all compared systems.
Article Search Article: issta26main-p29-p (type: Full Paper) doi:10.1145/3832094

XSearch: Explainable Code Search via Concept-to-Code Alignment
Yiming Liu, Ruofan Liu, Yun Lin, Zicong Zhang, Weiyu Kong, Pengnian Qi, Xiao Cheng, Weinan Zhang, Qianxiang Wang, and Linpeng Huang
(Shanghai Jiao Tong University, China; Shanghai Innovation Institute, China; National University of Singapore, Singapore; Huawei Technologies, China)
With the emergence of deep learning, semantic code search has been widely adopted in both academia and industry. These approaches embed natural-language queries and code snippets into a shared embedding space and retrieve results based on vector similarity. Despite their strong performance on benchmark datasets, they often suffer from poor explainability and generalization. Retrieved code may appear semantically similar yet miss critical functional requirements of the query, while providing no explanation of why the result was retrieved. Moreover, such failures become more severe under distribution shift, where models struggle to generalize to unseen benchmarks.
In this work, we propose XSearch, an intrinsically explainable code search framework. Our key insight is that, by relying on global embedding similarity, all existing retrievers inherently take an inductive view. They learn statistical patterns, rather than truly understand the query's functional requirements. Therefore, we address the problem by reformulating code search as a deductive concept alignment problem. At a high level, XSearch (i) identifies functional concepts in the query and (ii) explicitly aligns them with corresponding code statements. This explain-then-predict design not only produces inherent concept-level explanations, but also mitigates shortcut learning that harms out-of-distribution generalization. We train an encoder with explicit concept-alignment objectives and perform retrieval through explicit matching between query concepts and code statements. Experiments show that, when trained on CodeSearchNet with a small model size (GraphCodeBERT with 125M parameters), XSearch improves performance on out-of-distribution benchmarks from 0.02 to 0.33 (15×) over eight state-of-the-art retrievers, and consistently outperforms both encoder-based and decoder-based baselines with up to 7B parameters. A controlled user study further demonstrates that concept-alignment explanations enable users to accept or reject retrieved results both faster and more accurately. The source code is publicly available at https://github.com/code-philia/Xsearch.
Article Search Article: issta26main-p35-p (type: Full Paper) doi:10.1145/3832095

E4R-Reviewer: Effective and Explainable Automated Code Review via End-to-End Reasoning-Guided Alignment
Yifei Liu, Xizhi Hou, Li Yang, Huan Liu, Chen Zhu, Fengjun Zhang, and Chun Zuo
(Institute of Software at Chinese Academy of Sciences, China; University of Chinese Academy of Sciences, China; Sinosoft, China)
Code review is a key practice for ensuring software quality and maintainability. Despite progress in Automated Code Review (ACR), existing methods face two core challenges: (1) Isolated Task Modeling. Current approaches often model and optimize subtasks in ACR independently, ignoring the inherent logical order and internal dependencies among them which affects the effectiveness of ACR. (2) Lack of Explainability. At the task level, the absence of explanatory information in review comments increases developers’ cognitive load; at the model level, the black-box nature fundamentally undermines developer trust.
To address these challenges, we propose E4R-Reviewer, which improves the Effec­tiveness and Explain­ability of ACR through End-to-End Reasoning-guided alignment. For effectiveness, E4R-Reviewer unifies multiple fine-grained ACR subtasks into a single end-to-end reasoning process, enabling cross-task knowledge sharing and allowing the model to explicitly complete a reasoning chain that covers quality estimation, issue localization, issue classification, issue description, fix suggestion, and code refinement in one generation. Meanwhile, we adopt a Group Relative Policy Optimization (GRPO)-based reinforcement-learning alignment, treating the reasoning steps as optimizable intermediate objectives. We design subtask-specific rewards and integrate them via curriculum-inspired, multi-stage reward fusion that follows the real-world review workflow. For explainability, E4R-Reviewer produces reasoning process and structured review results covering all fine-grained ACR subtasks, improving the transparency and explainability of the review results.
Extensive evaluations on public, real-world datasets demonstrate that E4R-Reviewer significantly outperforms existing methods and achieves state-of-the-art performance: a 74.61% F1-score in quality estimation and +22.96% CodeBLEU in code refinement. Furthermore, Large Language Model (LLM) and human evaluation further confirm the superiority of E4R-Reviewer in terms of effectiveness and explainability.
Article Search Article: issta26main-p40-p (type: Full Paper) doi:10.1145/3832096

Automated Classification, Root Cause Analysis, and Repair Recommendations for Failed Mobile Testing by Specialized LLM
Chun Li, Fei Wang, Minxue Pan, Zhong Li, Mengliang Zeng, Bin Zhang, Xuejiao Yu, Boyun Wang, Kaijian Hua, and Xuandong Li
(Nanjing University, China; OPPO, China)
Engineers utilize test scripts to conduct testing on mobile applications to ensure software reliability. In industrial settings, failures in mobile application testing stem not only from faults in the program or scripts but also from other aspects, such as the testing environment, making it challenging to apply existing automated analysis methods in such settings. Furthermore, mobile testing in industrial contexts generates a large volume of artifacts, such as screenshots and logs, and involves diverse device models, varied test environments, and complex functionalities. All these factors make the manual analysis of failed mobile tests a process that is typically time-consuming, costly, and labor-intensive. Large language models (LLMs) with strong logical reasoning capabilities offer a novel perspective for automated analysis of failed mobile tests. However, due to their limited knowledge of mobile testing, general LLMs can only achieve sub-optimal effectiveness in these tasks. To address these challenges, we propose AnaDroid, a novel LLM training framework specifically tailored for analyzing failed mobile tests. AnaDroid performs feature preprocessing for failed mobile tests and trains the large language model through two novel, tailored training procedures. Specifically, AnaDroid first leverages bidirectional pre-training to instill the domain-specific knowledge and reasoning capabilities required for failed test analysis into the model. AnaDroid further employs preference optimization to enhance the model's capacity to capture critical information within the input context, thereby further improving its analytical effectiveness. Through extensive evaluations across six open-source LLMs of diverse architectures and scales on a large-scale industrial dataset of 284k failed mobile testing scripts, AnaDroid outperformed all four baselines across three analysis tasks. Furthermore, during a month-long deployment within a global company’s testing system, AnaDroid demonstrated its practical utility by successfully providing root cause and repair recommendations for 78% and 81% of the 11,304 failed tests, respectively.
Article Search Article: issta26main-p48-p (type: Full Paper) doi:10.1145/3832097

LLMutantKiller: Using Large Language Models to Generate Tests That Kill Mutants
Farideh Khalili, Aidan Domondon, Harshit Garg, and Frank Tip
(Northeastern University, USA; Amazon Web Services, USA)
The primary goal of mutation testing is to assess the quality of an application’s test suite. This is accomplished by introducing syntactic changes into a program and determining if any test failures occur for the resulting mutated program, commonly referred to as a mutant. If so, the mutant is said to be killed, confirming that the test suite is of sufficient quality to detect the introduced fault. A problem arises if a mutant does not impact the behavior of any test. Such a surviving mutant may occur for two reasons: either it involves a semantics- preserving program transformation or the test suite is not strong enough. Determining why a mutant survives often involves complex, non-local reasoning. This paper presents an LLM-based test generation technique for killing surviving mutants, implemented in a tool called LLMutantKiller. The technique is feedback-directed in the sense that if a test is produced that does not kill a given mutant, the LLM is re-prompted up to a specified number of times with scenario-specific feedback such as syntax errors, dependency violations, or execution logs (e.g., failing assertions) and asked to try again. We evaluate LLMutantKiller on 915 randomly selected surviving mutants produced by StrykerJS, a state-of-the-art mutation testing tool, across 13 open-source JavaScript/TypeScript applications. The results show that LLMutantKiller kills up to 95.3% of the surviving mutants classified as inducing behavioral changes and that it rarely produces invalid tests.
Article Search Artifacts Available Article: issta26main-p63-p (type: Full Paper) doi:10.1145/3832098

Artifact for "LLMutantKiller: Using Large Language Models to Generate Tests That Kill Mutants" (doi:10.5281/zenodo.21497478): LLMutantKiller This artifact is the public paper companion and replication package for LLMutantKiller: Using Large Language Models to Generate Tests that Kill Mutants. It contains the LLMutantKiller implementation, Docker packaging, mutation metadata, prepared benchmark provenance, complete stored experiment outputs, ...

Industrial Practice of LLM-Based Test Case Carving and Assertion Generation (Experience Paper)
Haozhen You, Zhen Dong, Jingjing Wang, Qiang Li, and Xin Peng
(Fudan University, China; ByteDance, China)
Enterprise regression testing for microservice systems is often constrained by incomplete or outdated documentation. In practice, QA engineers frequently rely on real execution traffic to reconstruct business scenarios; however, turning raw traffic into replayable regression tests with stable validation logic remains labor-intensive and error-prone.
This paper presents NL2Test, an end-to-end approach and tool that generates executable API regression tests from (i) a natural-language scenario description and (ii) a traffic capture recorded while executing the scenario. NL2Test addresses two coupled tasks: test case carving, which extracts a minimal replayable request sequence and reconstructs data dependencies so that dynamic values are bound from their responses rather than hard-coded; and assertion generation, which produces assertions aligned with business intent while avoiding non-deterministic fields and hallucinated paths. To improve reliability, NL2Test uses LLMs for semantic interpretation and constrained code synthesis, and uses deterministic algorithms for request filtering, dependency confirmation via value consistency, and assertion-path validation.
We evaluate NL2Test on 51 industrial regression scenarios extracted from a large consumer-facing Internet company. NL2Test achieves an exact-match rate of 82.4% (42/51), and produces a functionally usable draft in 98.0% (50/51) of scenarios when allowing minor post-edits. In a 9-month production deployment starting in March 2025, NL2Test generated 3,196 test cases with an overall code adoption rate of 85.4%. These results indicate that traffic-grounded generation with deterministic guardrails can substantially reduce manual effort while improving regression automation in complex microservice environments.
Article Search Article: issta26main-p73-p (type: Full Paper) doi:10.1145/3832099

Sifting the Noise: A Comparative Study of LLM Agents in Vulnerability False Positive Filtering
Yunpeng Xiong and Ting Zhang
(Monash University, Australia)
Static Application Security Testing (SAST) tools are essential for identifying software vulnerabilities, but they often produce a high volume of False Positives (FPs), imposing a substantial manual triage burden on developers. Recent advances in Large Language Model (LLM) agents offer a promising direction by enabling iterative reasoning, tool use, and environment interaction to refine SAST alerts. However, the comparative effectiveness of different LLM-based agent architectures for FP filtering remains poorly understood.
In this paper, we present a comparative study of three state-of-the-art LLM-based agent frameworks, i.e., Aider, OpenHands, and SWE-agent, for vulnerability FP filtering. We evaluate these frameworks using the vulnerabilities from the OWASP Benchmark and real-world open-source Java projects. We further conduct a focused post-cutoff C/C++ study using the strongest configuration to test contamination-free generalization and isolate key agentic capabilities. The experimental results show that LLM-based agents can remove the majority of SAST noise, reducing an initial FP detection rate of over 92% on the OWASP Benchmark to as low as 6.3% in the best configuration. On a real-world Java dataset, the best configuration of LLM-based agents can achieve an FP identification rate of up to 93.3% involving CodeQL alerts. However, the benefits of agents are strongly backbone- and CWE-dependent: agentic frameworks significantly outperform vanilla prompting for stronger models such as Claude Sonnet 4 and GPT-5, but yield limited or inconsistent gains for weaker backbones. On the post-cutoff OSS-Fuzz dataset, SWE-agent with Claude Sonnet 4 identifies 95.5% of FPs while maintaining 95.5% precision, compared with a 36.4% FP identification rate for vanilla prompting. Moreover, aggressive FP reduction can come at the cost of suppressing true vulnerabilities, highlighting important trade-offs. Finally, we observe large disparities in computational cost across agent frameworks.
Overall, our study demonstrates that LLM-based agents are a powerful but non-uniform solution for SAST FP filtering, and that their practical deployment requires careful consideration of agent design, backbone model choice, vulnerability category, and operational cost.
Article Search Artifacts Available Article: issta26main-p86-p (type: Full Paper) doi:10.1145/3832100

Artifact Package for Sifting the Noise: A Comparative Study of LLM Agents in Vulnerability False Positive Filtering (doi:10.5281/zenodo.21282004): This Zenodo record contains the ISSTA 2026 Artifact Evaluation package for the paper Sifting the Noise: A Comparative Study of LLM Agents in Vulnerability False Positive Filtering.

Datura: Progressive Red Teaming Testing for Tool Invocation Chain in LLM Agents
Yuchen Shao, Ziqun Bao, Yuheng Huang, Yuling Shi, Mingyu Weng, Yiwen Sun, Long Yang, Lei Ma, Ting Su, and Chengcheng Wan
(East China Normal University, China; Shanghai Innovation Institute, China; University of Tokyo, Japan; Shanghai Jiao Tong University, China; University of Alberta, Canada)
LLM agents that invoke external tools face critical safety vulnerabilities when malicious manipulations exploit their implicit trust in tool outputs and metadata. However, identifying these vulnerabilities through testing is challenging due to the need to bypass safety guardrails with semantically legitimate inputs, the difficulty in characterizing successful attacks amid implicit tool trust, and the requirement to maintain logical consistency across fragile state-dependent execution chains.
In this paper, we first conduct an empirical study to investigate how external tools influence agent reasoning. Guided by the findings, we propose Datura, an automated red teaming testing framework that exposes safety vulnerabilities through chained tool manipulation. Through a five-stage workflow, Datura dynamically generates test cases where each individual step appears legitimate yet collectively leads to harmful outcomes. We evaluate Datura across five LLMs and 740 safety-critical tasks under five defense settings, including real-world safety mechanisms. Under Model Alignment, Datura achieves 94.86--99.59% attack success rate (ASR), outperforming the strongest baseline by up to 25.27 percentage points. Under Prompt Refuge, Datura maintains 78.78--95.54% ASR, showing that progressive tool-chain manipulation remains effective even under prompt-level safeguards.
Article Search Artifacts Available Article: issta26main-p90-p (type: Full Paper) doi:10.1145/3832101

Artifact for Datura: Progressive Red Teaming Testing for Tool Invocation Chain in LLM Agents (doi:10.6084/m9.figshare.32744022.v4): This artifact accompanies the paper "Datura: Progressive Red Teaming Testing for Tool Invocation Chain in LLM Agents." It includes the source code of Datura, benchmark inputs, defense implementations, metric prompts, raw experimental outputs, scripts for package checking and evaluation, and a prebuilt Docker image for ...

Learning from the Test: Self-Referential Differential Testing for Deep RL Agents
Junda He, Jieke Shi, Zhou Yang, Mingfei Cheng, and David Lo
(Singapore Management University, Singapore; University of Alberta, Canada; Alberta Machine Intelligence Institute, Canada)
Deep Reinforcement Learning (DRL) has achieved significant success in complex decision-making problems. As DRL systems are increasingly deployed in real-world applications, ensuring their quality and reliability is paramount. Current works primarily focus on detecting safety-critical failures, often neglecting policy optimality, which can lead to reduced efficiency, user distrust, and economic losses. This oversight, compounded by the inherent “testing oracle problem” for optimality, leaves a significant gap in comprehensively evaluating DRL systems. To address this gap, we propose Delta (Differential Testing for DRL Agents), a novel and comprehensive framework that automatically identifies both safety-critical and optimality bugs in DRL agents. Delta employs a two-phase approach: (1) Safety Testing, where the Agent Under Test (AUT) is evaluated for catastrophic failures while collecting data from its decision-making policy, and (2) Optimality Testing, where this collected data from the prior phase is used to train a challenger agent via Offline Reinforcement Learning. Differential testing is then performed by comparing the challenger agent against the AUT; instances where the challenger achieves higher cumulative rewards indicate optimality issues in the AUT. We demonstrate Delta’s effectiveness across five environments. We investigate the effectiveness of three offline RL algorithms (BC, BCQ, and CQL) in generating challenger agents. Experimental results demonstrate that safety testing datasets are valuable for training competent DRL agents. Challenger agents trained with BCQ proved most effective for identifying optimality issues within the framework of Delta. Across the five environments, Delta uncovered an average of 2,518 optimality issues, outperforming the baseline methods by 50.2%.
Article Search Artifacts Available Article: issta26main-p101-p (type: Full Paper) doi:10.1145/3832102

Replication Package for ISSTA paper "Learning from the Test: Self-Referential Differential Testing for Deep RL Agents" (doi:10.6084/m9.figshare.29196884.v1): The DELTA (Differential Testing for Deep RL Agents) replication package provides a complete framework for automated testing of Deep Reinforcement Learning agents across five environments (CartPole, MountainCar, Acrobot, Hopper, and Walker2D). Located in the repository, the artifact combines per-environment ...

CLASScanner: Efficient C++ Class Recovery from Binaries Driven by Object Flow Graphs
Jiaming Wang, Gongming Wang, Songtao Yang, Xi Cao, and Chao Zhang
(Tsinghua University, China; Huazhong University of Science and Technology, China; Zhongguancun Laboratory, China; Science City (Guangzhou) Digital Technology Group Co. Ltd., China)
C++ class recovery is fundamental to reverse engineering, serving as a basis for critical downstream tasks such as vulnerability analysis, malware comprehension, and decompiler output optimization. Existing approaches face several challenges, including dependency on virtual function tables and a lack of support for non-polymorphic classes, dependency on high-quality test cases for dynamic analysis, and dependency on computationally expensive reasoning. Furthermore, existing rule-based techniques fail to recover class relationships in non-polymorphic classes, including inheritance and composition, thereby reducing fidelity to original program semantics. To address these problems, we propose CLASScanner, a novel approach for recovering C++ classes from stripped binaries. We design a data-flow abstraction, Object Flow Graph (OFG), to model the behaviors of objects across different contexts throughout their lifecycles. Driven by the OFG, CLASScanner identifies classes and recovers their attributes and methods. We further propose a progressive framework that synergizes static analysis with LLM-based reasoning to infer class inheritance and composition relationships. We evaluate CLASScanner on a dataset of real-world binaries comprising 167,982 functions. It achieved F1-scores of 95.4, 93.7, 75.4, 89.9, and 92.7 in recovering attributes, constructors, destructors, class inheritance, and class composition, respectively. Compared to state-of-the-art approaches, CLASScanner significantly improves the F1-scores while reducing runtime overhead, requiring only 10.3% of the execution time on average, making it promising for real-world reverse engineering tasks.
Article Search Article: issta26main-p107-p (type: Full Paper) doi:10.1145/3832103

Metamorphic Coverage
Jinsheng Ba, Yuancheng Jiang, and Manuel Rigger
(The Chinese University of Hong Kong, Shenzhen, China; National University of Singapore, Singapore)
Metamorphic testing is a widely used methodology that examines an expected relation between pairs of executions to automatically find bugs, such as correctness bugs. We found that code coverage cannot accurately measure the extent to which code is validated and mutation testing is computationally expensive for evaluating metamorphic testing methods. In this work, we propose Metamorphic Coverage (MC), a coverage metric that examines the distinct code executed by pairs of test inputs within metamorphic testing. Our intuition is that, typically, a bug can be observed if the corresponding code is executed when executing either test input but not the other one, so covering more differential code covered by pairs of test inputs might be more likely to expose bugs. While most metamorphic testing methods have been based on this general intuition, our work defines and systematically evaluates MC on five widely used metamorphic testing methods for testing database engines, compilers, and constraint solvers. The code measured by MC overlaps with the bug-fix locations of 50 of 64 bugs found by metamorphic testing methods, and MC has a stronger positive correlation with bug numbers than line coverage. MC is 4x more sensitive than line coverage in distinguishing testing methods' effectiveness, and the average value of MC is 6x smaller than line coverage while still capturing the part of the program that is being tested. MC required 359x less time than mutation testing. Based on a case study for an automated database system testing approach, we demonstrate that when used for feedback guidance, MC significantly outperforms code coverage, by finding 41% more bugs. Consequently, this work might have broad applications for assessing metamorphic testing methods and improving test-case generation.
Article Search Article: issta26main-p108-p (type: Full Paper) doi:10.1145/3832104

RPCSpecter: Detecting Blockchain RPC Bugs through a Specification-Driven, Constraint-Aware Fuzzing Approach
Yuming Xiao, Yuhong Nan, Zhijie Zhong, Mingxi Ye, and Zibin Zheng
(Sun Yat-sen University, China)
Blockchain Remote Procedure Calls (RPCs) serve as the primary interface for interaction between decentralized applications and blockchain networks. Despite their critical role, existing RPC implementations are prone to bugs that are often challenging to detect using traditional testing methods. In this paper, we introduce RPCSpecter, an automated framework for constraint-aware fuzz testing of blockchain RPC implementations. The core of RPCSpecter is a three-stage process: (1) Constraint Extraction, where implicit semantic dependencies from the documented RPC specifications are parsed and converted into executable constraints, (2) Constraint-Guided Mutation, which generates diverse and semantically valid test inputs based on these constraints, and (3) Bidirectional Assertion, which validates both valid and invalid RPC responses through dynamic checks and self-learning mechanisms. We evaluate RPCSpecter on both Ethereum and Solana, two predominant platforms in the Blockchain ecosystem, covering 6 clients, including Geth, Besu and Agave. The results show that RPCSpecter uncovers a total of 26 previously unknown bugs, including critical errors that are undetectable by existing fuzzers or manual testing, as well as multiple silent semantic inconsistencies. In particular, 4 of them have been acknowledged, and one of the bugs affecting three major Ethereum clients is confirmed as a vulnerability, with a $3,000 bounty award. Additionally, we demonstrate how RPCSpecter's constraint-driven approach significantly improves the efficiency and effectiveness of fuzz testing by systematically guiding mutation to explore boundary conditions and rare edge cases. Our research provides a more robust, scalable, and automated solution for enhancing the reliability and security of blockchain RPC implementations.
Article Search Article: issta26main-p118-p (type: Full Paper) doi:10.1145/3832105

TrapHunter: Exposing Covert Pathways in Trap Token Contracts
Yin Wu, Yixuan Liu, Yi Li, Chenyang Peng, Hao Wu, Ming Fan, Ting Liu, and Haijun Wang
(Xi'an Jiaotong University, China; Nanyang Technological University, Singapore)
Standardized token contracts (e.g., ERC-20) form the foundation of digital assets. However, attackers increasingly abuse this standardization to disguise malicious trap tokens. Unlike obvious violations, these contracts employ a strategy of "deceptive adherence": they strictly adhere to standard protocols to evade detection while embedding covert logic to defraud users. To address this, we first systematize the trap landscape by proposing a novel taxonomy derived from the intrinsic functional lifecycle of tokens (Generation, Circulation, Persistence, and Observation). We then propose TrapHunter, a framework designed to identify these traps and expose covert pathways within these deceptive contracts via intent deviation analysis. Specifically, TrapHunter introduces a unified semantic representation combining Abstract Behavior Trees (ABTs) and Augmented Path Graphs (APGs) to normalize intra-procedural syntax and reveal hidden execution paths driven by inter-procedural state dependencies. Crucially, it bridges the semantic gap by leveraging LLMs to reason about the behavioral intent of deviations from reference implementations, followed by fork-based dynamic validation to confirm exploitability. Experimental evaluation on 269 real-world contracts with three LLMs (DeepSeek, GPT, and Gemini) demonstrates that TrapHunter effectively detects all six categories of traps, achieving an average precision of 81.8% and recall of 85.4%, significantly outperforming state-of-the-art tools.
Article Search Article: issta26main-p151-p (type: Full Paper) doi:10.1145/3832106

SimiFuzz: Seed–Worker Scheduling for Parallel Fuzzing via Contextual Bandits
Yijia Guo, Zhiguo Ding, Hong Liang, Ming Zhong, Dandan Zhao, Xuhong Zhang, Bo Zhang, Shouling Ji, and Hao Peng
(Zhejiang Normal University, China; Zhejiang University, China; China Electric Power Research Institute, China)
Parallel fuzzing is now a standard way to scale vulnerability discovery, yet its efficiency is still limited by ineffective task allocation among workers. Existing approaches mainly aim to reduce conflicts; however, none considers the interaction between seeds and workers: the same seed can yield very different gains on different workers due to their divergent exploration states. As a result, parallel fuzzing can drift toward over-isolation that wastes shared states, or excessive overlap that duplicates effort.
To solve this problem, we present SimiFuzz, a context-aware scheduling framework that learns to assign seed–worker pairs online. SimiFuzz encodes each assignment with a compact context vector that jointly models seed characteristics, worker state, and seed–worker interaction. On top of this representation, SimiFuzz employs a LinUCB-based contextual bandit to score candidate pairs, balancing individual worker efficiency against group-level redundancy to maximize collective progress. To handle non-stationary fuzzing dynamics, SimiFuzz adopts a time-slice feedback mechanism that aggregates coverage gains within fixed intervals, combining globally new edges with cross-learning progress to form stable reward signals. We implement SimiFuzz on top of AFL++ and evaluate it on eight real-world targets. In 24-hour campaigns with 10 parallel instances, SimiFuzz improves average edge coverage by 11.76 % over FlexFuzz, the strongest baseline in coverage and unique vulnerability (VUL) count, achieves the highest final coverage on all evaluated targets, and uncovers 16 more unique vulnerabilities and 11 more CVEs than FlexFuzz.
Article Search Article: issta26main-p153-p (type: Full Paper) doi:10.1145/3832107

From Natural Language to Executable Properties for Property-Based Testing of Mobile Apps (Experience Paper)
Yiheng Xiong, Ting Su, Jingling Sun, Jue Wang, Qin Li, Geguang Pu, and Zhendong Su
(East China Normal University, China; Singapore Management University, Singapore; University of Electronic Science and Technology of China, China; Nanjing University, China; ETH Zurich, Switzerland)
Property-based testing (PBT) is a popular software testing methodology and is effective in validating the functionality of mobile applications (apps for short). However, its adoption in practice remains limited, largely due to the manual effort and technical expertise required to specify executable properties. In this experience paper, we propose a novel structured property synthesis approach that automatically translates property descriptions in natural language into executable properties, and implement it in a tool named iPBT. Our approach decomposes the problem into UI semantic grounding and executable property synthesis. It first builds an enriched widget context via multimodal LLMs to align visual elements with their functional semantics, and then uses an LLM with in-context learning to generate framework-specific executable properties. We evaluate with a closed-source LLM (GPT-4o) and an open-source LLM (DeepSeek-V3) on 160 diverse property descriptions across 20 apps (124 from an existing benchmark and 36 newly authored). iPBT achieves 95.0% (152/160) accuracy on both LLMs. Notably, an ablation study reveals that the enriched widget context contributes to an absolute improvement of up to 18.1% (from 76.9% to 95.0%). A user study with 10 participants demonstrates that iPBT reduces the time required to write executable properties by 56%, suggesting substantially lower manual effort. Furthermore, evaluations on 1,520 linguistically diverse paraphrases of the original property descriptions further confirm iPBT’s robustness, achieving 88.2% accuracy on GPT-4o and 87.8% on DeepSeek-V3.
Article Search Article: issta26main-p167-p (type: Full Paper) doi:10.1145/3832108

DDOR: Delta Debugging for Explainable Overrefusal Testing and Repair
Qinyan Zhou, Peixin Zhang, Jun Sun, Haonan Zhang, and Dongxia Wang
(Southeast University, China; AIDX TECH, Singapore; Singapore Management University, Singapore; Zhejiang University, China; Huzhou Institute of Industrial Control Technology, China)
While safety alignment and guardrails help large language models (LLMs) avoid harmful outputs, they can also induce overrefusal, i.e., unwarranted rejection of benign queries that merely appear risky. We present DDOR (Delta Debugging for OverRefusal), a fully automated and explainable framework for overrefusal testing and repair in a black-box setting, where only model inputs and outputs are accessible and internal safety mechanisms remain opaque. DDOR applies delta debugging to localize minimal refusal-triggering fragments (mRTFs) that provide phrase-level, explainable evidence for why a refusal occurs. Conditioned on these mRTFs, DDOR generates diverse, context-rich prompts and performs multi-oracle validation to filter intrinsically unsafe or ambiguous cases, producing scalable and model-specific overrefusal test suites (approximately 1K cases per model). Beyond evaluation, we further leverage localized mRTFs to perform targeted prompt repair, substantially reducing overrefusal while preserving the original intent and maintaining safety on genuinely harmful inputs. Overall, DDOR offers a practical end-to-end solution to both evaluate and mitigate overrefusal, improving LLM usability without sacrificing safety.
Article Search Article: issta26main-p170-p (type: Full Paper) doi:10.1145/3832109

Fairness Invariants: A Relational Approach to Explaining and Mitigating Fairness Bugs
Ranit Debnath Akash, Ashish Kumar, Gang Tan, and Saeid Tizpaz-Niari
(University of Illinois at Chicago, USA; Pennsylvania State University, USA)
Data-driven software systems are increasingly deployed in high-stakes socio-economic domains, from criminal justice to financial lending. However, these systems often exhibit individual discrimination—unjustified disparities in which a program yields different outcomes for similar individuals who differ only in their protected attributes (e.g., race, gender, age). While existing research has focused on detecting and quantifying these bugs, there remains a critical lack of principled mechanisms to explain and localize individual fairness bugs. Current explanation techniques are largely designed for single-input decisions rather than the relational nature of discrimination, which inherently involves a comparison between an original and a counterfactual pair.
We present Remi, a framework for the automated localization, explanation, and mitigation of individual discrimination. Inspired by loop-invariant synthesis in formal methods, we treat counterfactual fairness as a relational invariant discovery problem. We introduce a bidirectional relational explanation framework that learns over paired examples (x, x′) to identify regions of the input space where fairness is violated. Unlike traditional one-way implication pairs used in invariant inference, our approach enforces bidirectional constraints: requiring identical outcomes for both original and counterfactual samples. Remiutilizes three data-alignment techniques to infer interpretable rule-based models that act as ”fairness invariants.” These rules serve as guardrails to selectively block or relabel unfair predictions without requiring model retraining. Our evaluation on symbolic and neural network programs demonstrates that Remilocalizes ground-truth fairness bugs in over 83% of cases, significantly outperforming state-of-the-art baselines and reducing discriminatory decisions in black-box models by up to 70%.
Article Search Artifacts Available Article: issta26main-p172-p (type: Full Paper) doi:10.1145/3832110

Supplementary Materials and Reproduction Package of paper "Fairness Invariants: A Relational Approach to Explaining and Mitigating Fairness Bugs" (doi:10.6084/m9.figshare.33061115.v5): This artifact contains the reproduction package for REMI, a framework designed for the automated localization, explanation, and mitigation of individual discrimination (fairness bugs) in data-driven software systems. The package includes the source code for REMI's bidirectional relational explanation framework, the ...

SyzDiversity: Diversity-Guided Linux Kernel Fuzzing
Kun Hu, Jiaji Qin, Chaofeng Sha, Bihuan Chen, Shuoran Bai, Qicai Chen, Chenglin Wang, Xin Peng, and Wenyun Zhao
(Fudan University, China; Harbin Engineering University, China)
While coverage-guided kernel fuzzers have been proposed to uncover Linux kernel vulnerabilities, their code coverage and bug-finding capability are limited due to the lack of seed diversity, which is caused by the compounding effect of initial seed generation, seed scheduling, and seed mutation. To address this limitation, we propose a diversity-guided kernel fuzzer SyzDiversity. Specifically, to mitigate overvaluation of early seeds, it leverages proof-of-concept (PoC) seeds derived from real-world vulnerabilities as initial seeds, and further partitions these seeds into multiple communities. To improve diversity guidance in seed scheduling, it leverages a novel metric, community popularity rate (CPR), to model community diversity, and introduces a CPR-aware hierarchical Multi-Armed Bandit (MAB) algorithm that integrates CPR and code coverage as reward signals to prioritize the scheduling of diverse seed communities and seeds. Further, to efficiently populate sparse communities or break through community boundaries, it adopts a CPR-guided seed mutation strategy that adaptively allocates higher mutation frequencies to communities that are more conducive to the diversity evolution of the seeds. Our extensive experiments on Linux kernel versions v5.15 and v6.14 have demonstrated that SyzDiversity improves code coverage and bug-finding capability by 17.2% and 6.4×, respectively, compared to the state-of-the-art kernel fuzzers. It has discovered 32 unique new vulnerabilities, with 12 of them confirmed.
Article Search Article: issta26main-p188-p (type: Full Paper) doi:10.1145/3832111

Sakura: An Approach for Generating Complex Tests from Natural Language Test Descriptions
Tyler Stennett, Rangeet Pan, Bridget McGinn, Alessandro Orso, and Saurabh Sinha
(Georgia Institute of Technology, USA; IBM Research, USA; University of Georgia, USA)
Testing is a core activity in software development, and research on its automation has spanned several decades. Most existing approaches focus on generating unit tests for individual methods, validating isolated API endpoints, or targeting user interface (UI) layers. However, for non-API and non-UI tests, automated test generators typically exercise a single focal method. Recent empirical evidence shows a substantial gap between such generated tests and developer-written tests, which often span several focal classes and methods, involve multi-step call sequences, and contain chained assertions—characteristics that current automated approaches fail to capture. To address this gap, we propose generating tests from natural language (NL) descriptions of developer intent. NL provides an expressive and accessible medium for specifying complex test scenarios and functional intent. We present Sakura, the first agent-based framework for generating structurally complex test cases from NL descriptions. Sakura decomposes NL descriptions into structured blocks and processes them using a multi-agent system consisting of a localization agent that grounds test steps in concrete application code via static analysis, a composition agent that synthesizes compilable test code and iteratively refines it using execution feedback, and a supervisor agent that coordinates agent interactions. To evaluate Sakura, we curate a novel dataset of NL test descriptions at three levels of abstraction, reflecting different end-user personas, systematically derived from developer-written tests in Apache Commons projects. Across 20 applications and 1,464 test scenarios, Sakura substantially outperforms off-the-shelf agentic tools such as Gemini CLI instantiated with multiple LLMs. Specifically, Sakura achieves 50–78% higher test compilability and 38–66% higher coverage overlap with ground-truth tests compared to baselines using the same models. Moreover, Sakura paired with small open-source models such as Devstral Small 2 and Qwen3-Coder outperforms Gemini CLI using large proprietary models, while also being more cost-effective.
Article Search Artifacts Available Article: issta26main-p192-p (type: Full Paper) doi:10.1145/3832112

Reproduction Package for Article "Sakura: An Approach for Generating Complex Tests from Natural Language Test Descriptions" (doi:10.5281/zenodo.21710100): This artifact is the reproduction package for the ISSTA 2026 paper, "Sakura: An Approach for Generating Complex Tests from Natural Language Test Descriptions." It contains the full implementation of Sakura, the evaluation dataset, the pre-computed results reported in the paper, and the scripts to regenerate every ...

To Run or Not to Run: Analyzing the Cost-Effectiveness of Code Execution in LLM-Based Program Repair
Zhihao Lin, Junhua Zhu, Mingyi Zhou, Xin Wang, Zhensu Sun, Renyu Yang, David Lo, and Li Li
(Beihang University, China; Wuhan University, China; Singapore Management University, Singapore)
LLM-based agents for program repair are increasingly built on a “generate-run-revise” paradigm, iteratively executing tests to evaluate and refine patches. This execution-based approach has become standard practice in state-of-the-art systems. However, executions can be time-consuming and expensive, yet their impact on these agents remains underexplored. In this paper, we conduct a two-stage empirical study of execution behavior in LLM-based program repair. To characterize execution behavior at scale, we first analyze 7,745 agent traces from SWE-bench leaderboard submissions. Second, we evaluate 3,000 end-to-end repair attempts across 200 SWE-bench instances and three agents (Claude Code, Codex, and the open-source OpenCode) under four execution paradigms, which allows for a fine-grained comparison of performance and cost. Our analysis reveals three key observations: (1) Code execution is used across all agents and models analyzed, with an average of 8.8 test runs per task. Execution behavior varies substantially across agents and models, with frequency ranging from 2 to 19 per task, and late-stage executions (66–100% of conversation) consistently achieve higher success rates than early-stage ones (57.9% average). (2) Execution restrictions have little effect on repair success: On commercial agents with SOTA models, the resolve-rate gap between Prohibited and Unrestricted is only 1.25pp (not statistically significant, p > 0.05). The corresponding value for open-source OpenCode with Qwen2.5-Coder-32B is approximately 0pp, with equivalence holding under both prompt-level and tool-level enforcement of the restriction. Prohibited saves 56–62% of tokens and 48–54% of wall-clock time on Claude Code, and removes the need to maintain per-repository test environments. (3) Execution benefit is concentrated rather than uniform. For commercial agents, 54–66% of cases complete in a single edit, localization accuracy under Prohibited is over 95%, and 81–100% of failed cases pass agent-executed validation but fail the official evaluation. OpenCode with Qwen2.5-Coder-32B shows another failure mode: it retries more frequently and only 11% of its failed cases pass self-validation. These patterns suggest that current agents apply execution indiscriminately, paying its cost on instances where it provides little benefit. Execution, therefore, should be treated as a resource with an explicit cost-benefit tradeoff, not a default capability.
Article Search Artifacts Available Article: issta26main-p200-p (type: Full Paper) doi:10.1145/3832113

To Run or Not to Run artifact for Analyzing the Cost-Effectiveness of Code Execution in LLM-Based Program Repair (doi:10.5281/zenodo.21787572): This artifact is the public reproduction package for the ISSTA 2026 paper “To Run or Not to Run: Analyzing the Cost-Effectiveness of Code Execution in LLM-Based Program Repair”. It contains the experiment runner, execution-policy prompts, Docker and SWE-bench integration scripts, compact benchmark metadata, analysis ...

LLM-Based Repair of Static Nullability Errors
Nima Karimipour, Pascal Joos, Michael Pradel, Martin Kellogg, and Manu Sridharan
(University of California at Riverside, USA; CISPA Helmholtz Center for Information Security, Germany; New Jersey Institute of Technology, USA)
Modern Java projects increasingly adopt static analysis tools that prevent null-pointer exceptions by treating nullness as a type property. However, integrating such tools into large, existing codebases remains a significant challenge. While annotation inference can eliminate many errors automatically, a subset of residual errors—typically a mix of real bugs and false positives—often persists and can only be resolved via code changes. Manually addressing these errors is tedious and error-prone. Large language models (LLMs) offer a promising path toward automating these repairs, but naively prompted LLMs often generate incorrect, contextually inappropriate edits. We present NullRepair, a system that integrates LLMs into a structured workflow for resolving the errors from a nullability checker. NullRepair’s decision process follows a flowchart derived from manual analysis of 200 real-world errors. It leverages static analysis to identify safe and unsafe usage regions of symbols, using error-free usage examples to contextualize model prompts. Patches are generated through an iterative interaction with the LLM that incorporates project-wide context and decision logic. Our evaluation on 12 real-world Java projects shows that NullRepair resolves 63% of the 1,119 nullability errors that remain after applying a state-of-the-art annotation inference technique. Unlike two baselines (single-shot prompt and mini-SWE-agent), NullRepair also largely preserves program semantics, with all unit tests passing in 10/12 projects after applying every edit proposed by NullRepair, and 98% or more tests passing in the remaining two projects.
Article Search Artifacts Available Article: issta26main-p228-p (type: Full Paper) doi:10.1145/3832114

Reproduction Package for "LLM-Based Repair of Static Nullability Errors" (doi:10.5281/zenodo.20645902): Contains the NullRepair code, as well as all evaluation scripts, benchmarks, and evaluation results. The ready built Docker image can be used to reproduce all results from the paper.

How Much Static Structure Do Code Agents Need? A Study of Deterministic Anchoring
Zhihao Lin, Mingyi Zhou, Yizhuo Yang, and Li Li
(Beihang University, China)
LLM-based code agents navigate repositories through keyword search but lack access to the structural relationships, such as call graphs, inheritance hierarchies, and configuration dependencies, that organize software systems. This limitation can make agent navigation stochastic and difficult to reproduce across runs. We investigate whether lightweight static analysis can provide deterministic anchors: stable structural facts injected as plain-text comments to guide probabilistic exploration. Starting from Codex as a strong grep-first baseline, we systematically vary the granularity and directionality of structural annotations and measure their effects on localization, navigation behavior, and run-to-run stability. Lightweight call and inheritance topology improves function-level localization by 2.2 percentage points at Func@5 and shortens trajectories by 1.6 interaction rounds. The optimal annotation design depends on repository scale: denser semantic annotations provide diminishing returns, while hub-heavy projects benefit from inverse-only links that avoid excessive forward-edge noise. Structural tags also increase the link-following rate from 0.15–0.18 to 0.21–0.24, reduce run-to-run variance by roughly half, and improve single-run reliability by 3.4 percentage points at Pass@1 on medium-scale repositories, at the cost of approximately 10% more input tokens. These results show that static structure improves code-agent navigation by making exploration more disciplined and reproducible. They support a topology-first design: use lightweight bidirectional structure for medium-scale repositories, prune forward edges in hub-heavy repositories, and reserve dense annotations for tasks involving implicit dependencies.
Article Search Artifacts Available Article: issta26main-p241-p (type: Full Paper) doi:10.1145/3832115

CodeAnchor artifact for How Much Static Structure Do Code Agents Need? A Study of Deterministic Anchoring (doi:10.5281/zenodo.21787482): This artifact is the public reproduction package for the ISSTA 2026 paper “How Much Static Structure Do Code Agents Need? A Study of Deterministic Anchoring”. It contains the CodeAnchor experiment runners, evaluation scripts, compact evaluation summaries, example prompts, and paper-facing figures. The artifact ...

The Fix Is Right at Hand: Fixing Incompatibility Errors Guided by Library Knowledge for Automatic Library Upgrade
Zhuotong Zhou, Susheng Wu, Junpeng Zhao, Bihuan Chen, YenQin Hoo, Yiheng Huang, Yiheng Cao, and Xin Peng
(Fudan University, China)
Third-party libraries (TPLs) play critical roles in modern software development. Upgrading them is crucial for enhanced security and functionality, but often introduces incompatibility errors, caused by breaking changes in library APIs, in client code. Existing approaches rely on predefined migration patterns or API recommendation heuristics, which suffer from limited pattern coverage and ignore the usage context of broken API, leading to incorrect or incomplete fixes. To address these limitations, we propose Librarian, a novel LLM-based approach to automatically fix incompatibility errors when upgrading a dependent library in a client project. The core idea of Librarian is to extract context-aware fix hints from the library codebase, serving as semantic few-shot examples, enabling LLM to generate fixes without relying on predefined patterns. Since LLM may generate an incorrect or incomplete fix, Librarian performs fix refinement based on compilation feedback from the client project. Our evaluation has demonstrated that Librarian achieves a fixing success rate of 84.2%, outperforming the state-of-the-arts by at least 45.3%. Our evaluation has also indicated the practical usefulness of Librarian in fixing incompatibility errors in 32 real-world projects.
Article Search Article: issta26main-p242-p (type: Full Paper) doi:10.1145/3832116

SWE-PDB: Teaching LLMs to Leverage Debugging Tools via Agentic Training
Jiaxing Liu, Xing Hu, and Xin Xia
(Zhejiang University, China; Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security, China)
Interaction with debugging tools enables large language models (LLMs) to reason over concrete runtime states, rather than relying solely on static analysis of source code. Specifically, with the help of a debugger, an LLM-based agent can observe actual program execution by inspecting intermediate variable states and stepping through the control flow. These runtime observations enable the model to better understand program behavior and identify the root causes of bugs. Despite these advantages, using debuggers correctly and effectively remains challenging for many models because debugger interaction is inherently stateful and requires executing complex, long-horizon action sequences. As a result, models often exhibit unproductive interactions in which the debugger is underutilized or even disrupts the debugging process.
To address this challenge, we propose SWE-PDB, the first training-based framework that teaches LLMs to leverage debuggers for interactive debugging and program repair. Our approach constructs large-scale buggy Python instances with verified failing tests from diverse sources and synthesizes multi-turn interactive debugging trajectories that follow structured debugging workflows. To ensure data quality, we apply multi-stage trajectory filtering and refinement, and train models using agentic supervised fine-tuning to learn effective debugger behaviors, followed by agentic reinforcement learning with rule-based rewards to improve generalization and promote more strategic debugger usage. Extensive evaluations across diverse benchmarks demonstrate substantial gains. In particular, SWE-PDB-14B achieves 38.0% accuracy on SWE-bench Verified with complete test suites, more than tripling the base model’s performance, while improving interaction efficiency and exhibiting robust, meaningful debugger usage.
Article Search Article: issta26main-p266-p (type: Full Paper) doi:10.1145/3832117

PackMonitor: Towards Zero Package Hallucinations via Decoding-Time Monitoring
Xiting Liu, Yuetong Liu, Yitong Zhang, Jia Li, and Shi-Min Hu
(Tsinghua University, China; Beihang University, China)
As Large Language Models (LLMs) are increasingly integrated into software development workflows, their trustworthiness has become a critical concern. However, in dependency recommendation scenarios, the reliability of LLMs is undermined by widespread package hallucinations, where models often recommend hallucinated packages. Recent studies have proposed a range of approaches to mitigate this issue. Nevertheless, existing approaches typically merely reduce hallucination rates rather than eliminate them, leaving persistent software security risks.
In this work, we argue that package hallucinations are theoretically preventable based on the key insight that package validity is decidable through finite and enumerable authoritative package lists. Building on this, we propose PackMonitor, the first approach capable of fundamentally eliminating package hallucinations by continuously monitoring the model's decoding process and intervening when necessary. To implement this in practice, PackMonitor addresses three key challenges: (1) determining when to trigger intervention via a Context-Aware Parser that continuously monitors model outputs and selectively activates intervening only during installation command generation; (2) resolving how to intervene by employing a Package-Name Intervenor that strictly limits the decoding space to an authoritative package list; and (3) ensuring monitoring efficiency through a DFA-Caching Mechanism that enables scalability to millions of packages with negligible overhead. Extensive experiments on five widely used LLMs demonstrate that PackMonitor is a training-free, plug-and-play solution that consistently reduces package hallucination rates to zero while maintaining low-latency inference and preserving original model capabilities.
Article Search Article: issta26main-p273-p (type: Full Paper) doi:10.1145/3832118

TensorLock: Recovering Model Dependency for Model Supply Chain
Susheng Wu, Ziqian Chen, Chengyuan Li, Kaifeng Huang, Zekai Chen, Yijian Wu, Bihuan Chen, Yiheng Cao, Zhuotong Zhou, Yiheng Huang, and Xin Peng
(Fudan University, China; Tongji University, China)
The public models on the model hosting platforms have undergone exponential growth, allowing developers to build upon existing models rather than training from scratch. These models are continuously reused, modified, and re-distributed similar to traditional software components, breeding a dense and rapidly evolving model supply chain. However, while enjoying the benefits of model reuse, developers also inherit supply chain risks ranging from legal liabilities to security threats. To mitigate these risks, a well-established model dependency graph can significantly benefit supply chain risk governance. Unfortunately, although model hosting platforms offer mechanisms for dependency disclosure, such declarations are optional and frequently missing.
To address this challenge, we propose a novel model dependency recovering framework Tensorlock. It works in two phases; i.e., (1) model clustering, and (2) type-aware dependency identification within these clusters. In the first phase, Tensorlock performs connectivity-based clustering to accommodate the open-ended dependency topology, grouping models with dependency relations. In the second phase, Tensorlock employs a divide-and-conquer strategy, leveraging distinct type-specific fingerprints to first identify data-free dependencies (Quantization and Merging), and then resolve data-driven Fine-Tuning dependencies. Our evaluation demonstrates that Tensorlock substantially outperforms state-of-the-art approaches, achieving an ARI of 0.96 in clustering and a DF1 of 0.82 in dependency identification, improving over the best baselines by at least 39% and 193%, respectively. Additionally, we apply Tensorlock to 289 supposedly isolated models and recover 189 previously missing model dependencies, with 42 model authors confirming our findings.
Article Search Article: issta26main-p276-p (type: Full Paper) doi:10.1145/3832119

SEER: Self-Enhancing Chain-of-Thought Compression for Reasoning Models
Kerui Huang, Shuhan Liu, Xing Hu, Tongtong Xu, Lingfeng Bao, and Xin Xia
(Zhejiang University, China; Nanjing University, China; Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security, China)
Chain-of-Thought (CoT) prompting can substantially improve the reasoning ability of large language models (LLMs), but it often comes with high inference cost due to long and poorly controlled reasoning traces. This overhead is particularly problematic in software engineering tasks (e.g., code generation), where both latency and output reliability matter.
To better understand this trade-off, we conduct an empirical study on widely used code generation benchmarks and observe that many modern reasoning models produce excessively verbose CoTs (often thousands of tokens), which frequently leads to truncation and unstable generation. Using a strict n-gram repetition detector, we find that most observed truncations are associated with degenerate looping behaviors. In addition, a HumanEval/129 case study shows that failed generations can be longer than successful ones, suggesting limited returns from overlong reasoning.
Motivated by these findings, we propose SEER (Self-Enhancing Efficient Reasoning), a self-enhancing framework for adaptive CoT compression. improves the conciseness of reasoning while preserving output quality, without relying on external compression tools. refines self-generated CoT data via Best-of-N sampling to suppress looping and redundant traces, then applies a lightweight, data-driven filter to encourage concise yet correct reasoning. It then fine-tunes the model on the filtered data to internalize concise reasoning behaviors. Across four software engineering benchmarks on the evaluated DeepSeek-R1-Distill-Qwen-7B backbone, reduces CoT length by 34.6% on average while improving task performance, with reduced truncation and fewer reasoning loops.
Article Search Article: issta26main-p277-p (type: Full Paper) doi:10.1145/3832120

NSync: Automated Cloud Infrastructure-as-Code Reconciliation with AI Agents
Zhenning Yang, Hui Guan, Victor Nicolet, Brandon Paulsen, Joey Dodds, Daniel Kroening, and Ang Chen
(University of Michigan, USA; Amazon, USA)
Cloud infrastructure is managed through a mix of interfaces—traditionally, cloud consoles, command-line interfaces (CLI), and SDKs are the tools of choice. Recently, Infrastructure-as-Code/IaC frameworks (e.g., Terraform) have quickly gained popularity. Unlike conventional tools, IaC frameworks encode the infrastructure in a ”source-of-truth” configuration. They are capable of automatically carrying out modifications to the cloud—deploying, updating, or destroying resources—to bring the actual infrastructure into alignment with the IaC configuration. When IaC frameworks are used together with consoles, CLI, or SDKs, IaC is unaware of changes through these non-IaC interfaces, and the IaC configuration no longer captures the intended state. This is called infrastructure drift. IaC frameworks will revert non-IaC changes based on the outdated IaC configuration, leading to misconfigurations or failures.
We propose NSync, an automated system for IaC reconciliation, which aims to propagate out-of-band changes back to the IaC program in the form of an update. Our key insight is that infrastructure changes via IaC, consoles, CLI, or SDK eventually all occur via cloud API invocations—the lowest layer for cloud management operations. Hence, NSync gleans insights from API traces to detect drift (i.e., non-IaC changes) and reconcile it (i.e., update the IaC configuration to capture the changes). This is a challenging task—identifying the intended change from low-level, noisy API traces is not easy; moreover, because of the criticality of cloud infrastructure, NSync cannot directly test the synthesized updates in a live environment. NSync addresses these challenges using an agentic design. It infers high-level infrastructure change intent from cloud API sequences with the help of LLMs, and synthesizes targeted IaC updates using domain-specific context management with customized agent tooling; it further maintains an evolving knowledge base of past successful reconciliation runs, reusing prior insights to achieve higher accuracy on future tasks. In addition to system design, we contribute a novel evaluation pipeline for injecting drift into cloud infrastructure and assessing reconciliation attempts, by sourcing scenarios from authoritative cloud operation examples and transplanting them into an IaC-centric framework. Experiments across five real-world Terraform projects and 372 drift scenarios show that NSync outperforms the baseline both in terms of accuracy (from 0.71 to 0.97 pass@3) and token efficiency (1.47× improvement).
Article Search Article: issta26main-p281-p (type: Full Paper) doi:10.1145/3832121

Supplementary Material: This supplementary material provides additional details on NSync’s benchmark construction and evaluation. It documents the selection of AWS Automation runbooks, project-specific drift mutations, validation procedures, and per-project scenario coverage, with additional runtime and code-maintainability results.

Test vs Mutant: Adversarial LLM Agents for Robust Unit Test Generation
Pengyu Chang, Yixiong Fang, Silin Chen, Yuling Shi, Beijun Shen, and Xiaodong Gu
(Shanghai Jiao Tong University, China)
Software testing is a critical, yet resource-intensive phase of the software development lifecycle. Search-based approaches typically achieve high coverage but produce tests with low readability, whereas large language model (LLM)-based methods generate more human-readable tests but often suffer from low coverage and compilability. While the majority of research efforts have focused on improving test coverage and readability, comparatively less attention has been paid to enhancing the robustness of bug detection. To address this gap, we propose AdverTest, a novel adversarial framework for LLM-powered test case generation that pairs a test case generation agent T with a mutant generation agent M: M persistently creates mutants "hacking" the blind spots of T's current test suite, whileT iteratively refines its tests to "kill" the challenging mutants, with the interaction guided by both coverage and mutation scores. Experimental results on Defects4J show that our approach improves fault detection rates by 8.56% over the best existing LLM-based methods and by 50.20% over EvoSuite, while remaining competitive on line and branch coverage.
Article Search Article: issta26main-p285-p (type: Full Paper) doi:10.1145/3832122

Guarding the Lifeline: A First Look and Automated Defect Diagnosis for ROS Central Index
Weijie Sun, Huiyan Wang, Ying Wang, and Chang Xu
(Nanjing University, China; Northeastern University, China)
The Robot Operating System (ROS) relies on a centralized dependency index, the rosdistro central index, to manage packages across its heterogeneous software ecosystem, which integrates independently evolving Operating System (OS) repositories for system libraries, ROS repositories for domain-specific support, and Programming Language (PL) repositories for functional modules. While this design enables portability, it introduces a critical source of fragility since ROS dependency management depends entirely on this manually curated, static index that must map packages across independently evolving, multi-source repositories. This leads to persistent defects in the central index, such as missing, incorrect, or outdated installation rules, which undermine the reliability of ROS dependency management. To address this problem, we conducted the first in-depth empirical study of 863 real-world maintenance cases involving the ROS central index. We categorize defects as either coverage or correctness defects, identify their underlying structural causes, and demonstrate that the primary bottleneck in manual maintenance is the difficulty of identifying equivalent packages across repositories. Motivated by these findings, we propose RosdepAuditor, an automated auditing framework that introduces a cross-repository mapping mechanism with hybrid scoring to infer package equivalence and detect defects. Evaluated on a ground-truth dataset, RosdepAuditor achieves 94.7% mapping accuracy without recommending non-existent packages, outperforming existing pattern-based and upstream-based approaches as well as leading large language models (LLMs). When applied to the live index, it uncovered 3,249 potential defects across 2,233 entries, 46 of which have been confirmed and fixed, demonstrating its practical usefulness in strengthening ROS dependency management.
Article Search Artifacts Available Article: issta26main-p297-p (type: Full Paper) doi:10.1145/3832123

Guarding the Lifeline: A First Look and Automated Defect Diagnosis for ROS Central Index (doi:10.5281/zenodo.21280924): RosdepAuditor Artifact Package This archive accompanies the paper "Guarding the Lifeline: A First Look and Automated Defect Diagnosis for ROS Central Index." RosdepAuditor is a tool for auditing ROS's central dependency index. It infers equivalent packages across repositories, then diagnoses missing, incomplete, ...

Scitix: Scalable Constraint-Based Type Inference for Code Snippets with Missing Types
Yiwen Dong, Zhenyang Xu, Yongqiang Tian, Edward Lee, Ondřej Lhoták, and Chengnian Sun
(University of Waterloo, Canada; Monash University, Australia; University of Toronto at Scarborough, Canada)
Code snippets commonly appear in online developer communities, documentation, and LLM-assisted workflows to communicate ideas and algorithms. However, contextual information, like dependencies and the exact types, are often missing in code snippets, which makes their reuse difficult. Some of the most successful automated techniques use logical constraints to infer the types and dependencies, but they do not work in practice because they require an exact knowledge base that contains all possible dependencies and exact types. However, such a knowledge base is both computationally expensive for constraint solving and impossible to achieve in the presence of missing types (e.g., user-defined types) in code snippets. To this end, this paper proposes a novel, scalable technique named Scitix. Our insight is two-fold. First, inspired by gradual typing’s use of an unknown type, we represent certain missing types as Any, ignoring such types during constraint solving, improving performance and scalability. Second, our novel, iterative constraint-solving approach saves on computation and skips constraints involving missing types. Our extensive evaluations show that our insights improve both performance and scalability compared to SnR (the state of the art). Specifically, Scitix achieves F1-scores of 94.8% and 86.8% on Stack Overflow and generated code snippets, respectively, using a large knowledge base of over 3,000 jars. In contrast, SnR consistently times out, yielding near 0% F1. Even with the smallest knowledge base, where SnR does not time out, Scitix reduces the number of errors by 77% and 45% compared to SnR. Compared to state-of-the-art large language models (LLMs) like GPT-4o and the LLM-based ZS4C, Scitix improves F1-score by 76.8% and 35.4%, respectively. Scitix’s strong performance highlights its potential as a practical technique for type inference in real-world code snippets.
Article Search Artifacts Available Article: issta26main-p303-p (type: Full Paper) doi:10.1145/3832124

Scitix: Scalable Constraint-Based Type Inference for Code Snippets with Missing Types (Artifact) (doi:10.5281/zenodo.21018232): This replication package contains the source code, benchmark results, Docker files, and scripts for running the benchmarks for Scitix.

Beyond Similarity Scores: Evidence-Based Third-Party Library Detection for C/C++ Binaries
Chengyue Liu, Zhengzi Xu, Lyuye Zhang, Jiahui Wu, Kaixuan Li, and Yang Liu
(Nanyang Technological University, Singapore; Imperial Global Singapore, Singapore)
Detecting third-party libraries (TPLs) in C/C++ binaries is essential for software supply chain security, enabling vulnerability identification and license compliance. Existing methods predominantly rely on similarity matching: extracting features from binaries and comparing them against library databases. However, similarity scores alone cannot reliably determine library presence. Low similarity causes false negatives when matchable features are limited. More critically, high similarity does not guarantee accuracy: libraries often share features through shared dependencies, forks, or similar functionality, causing multiple candidates to match even when only one is present. These issues suggest that similarity matching is effective for narrowing candidates but insufficient as the final decision mechanism. Rather than relying solely on similarity scores, reliable detection requires multi-source evidence to verify each candidate.
To this end, we propose BLADE, which reframes TPL detection as evidence-based candidate verification. Instead of relying on similarity scores to make final decisions, BLADE retrieves candidates broadly to mitigate false negatives, and then collects evidence from multiple sources, which an LLM analyzes through structured verification workflows to filter false positives: first confirming candidates with clear identity markers, then systematically checking remaining candidates against common false positive patterns. To evaluate BLADE, we build the largest C/C++ binary TPL benchmark to date, comprising 3,403 binaries and 1,016 libraries. Results show that BLADE achieves 97.60% precision and 93.74% recall (F1: 95.63%), improving F1-score by 41.83 percentage points over the best baseline. The average cost is $0.0378 per binary. BLADE has been deployed in a commercial software composition analysis product, demonstrating practical feasibility at scale.
Article Search Article: issta26main-p308-p (type: Full Paper) doi:10.1145/3832125

Defusing Logic Bombs in Symbolic Execution with LLM-Generated Ghost Code
Dimitrios Stamatios Bouras and Sergey Mechtaev
(Peking University, China)
Symbolic execution is a powerful program analysis technique, but its effectiveness is fundamentally limited by solver-hostile program fragments, complex numerical reasoning, and unbounded heap structures. Recent work proposed replacing constraint solvers with large language models (LLMs) to bypass these limitations, but such approaches struggle to analyze real-world codebases, where deep execution paths require globally consistent reasoning across many interacting constraints. We present Gordian, a hybrid symbolic execution framework that uses LLMs selectively to generate lightweight ghost code that aids an SMT solver in handling solver-hostile code fragments, while preserving its precise, global reasoning capability. In particular, we propose three types of ghost code: (1) inversion of difficult code fragments with iterative bidirectional constraint propagation, (2) modeling via solver-friendly surrogates while preserving relevant behavior, and (3) semantic partitioning of unbounded heap spaces. We implemented Gordian on top of the KLEE symbolic execution engine and evaluated it on synthetic “logic bombs” capturing distinct symbolic reasoning challenges, a popular mathematical library FDLibM, and four structured-input programs (libexpat, jq, bc and libyaml). Across benchmarks, Gordian improves coverage by 28.5–115.2% over traditional symbolic execution baseline and by 74.1–189.8% over LLM-based symbolic execution baselines, while reducing LLM token usage by an average of 91–96%. This highlights the practicality and effectiveness of this approach in real-world settings.
Article Search Article: issta26main-p310-p (type: Full Paper) doi:10.1145/3832126

Tracing the Shadows: Automatic Tracking and Analysis of Crypto Money Laundering via Transaction Semantic Analysis
Hao Wu, Haijun Wang, Shangwang Li, Yin Wu, Ming Fan, Ting Liu, and Xiapu Luo
(Xi'an Jiaotong University, China; Hong Kong Polytechnic University, Hong Kong)
With the rapid advancement of decentralized finance (DeFi), security incidents related to cryptocurrency have become increasingly prevalent. After such incidents, attackers typically attempt to rapidly move stolen assets, concealing the origin of illicit funds and ultimately converting them into fiat currency.However, existing anti-money laundering (AML) methods struggle to cope with the semantic complexity of DeFi transactions. They either rely heavily on low-level token transfers, or perform protocol-agnostic money flow analysis, failing to capture the high-level intent of transactions. This limitation results in misleading tracing paths with substantial noise and fails when laundering activities span multiple blockchains.
In this paper, we propose AMLGuard, a semantic-aware AML framework for account-based blockchains. AMLGuard tracks illicit fund flows from known malicious addresses by performing semantic analysis on complex DeFi transactions, enabling accurate and continuous laundering tracking. Given a complex transaction, AMLGuard combines static rule-based analysis with retrieval-augmented large language model (LLM) reasoning to infer implicit DeFi semantics, transforming raw transaction data into high-level semantic representations. Furthermore, for cross-chain transactions where laundering intent is not explicitly exposed, AMLGuard parses transaction parameters and performs argument parsing to recover cross-chain semantics, enabling seamless tracking across ledgers. Based on inferred semantics, AMLGuard abstracts each transaction into a DeFi Semantic Unit (DSU). These DSUs are analyzed and composed iteratively to update account states, expand the tracing frontier, and ultimately construct the illicit fund-flow topology. We evaluate the effectiveness of AMLGuard on 82 real-world laundering cases, involving illicit assets worth over $1 billion. Specifically, AMLGuard reconstructs compact illicit fund-flow topologies with destination precision of 94.4% and 87.6%, while achieving the highest address recall of 98.4% and 95.8% and destination recall of 94.1% and 93.8% on single-chain and cross-chain datasets. Furthermore, a case study demonstrates that AMLGuard can assist real-world AML investigation, substantially reducing analysis time and effort.
Article Search Article: issta26main-p329-p (type: Full Paper) doi:10.1145/3832127

PropCov: Effective Coverage Reporting for Property-Based Testing
Jesse Coultas, Joseph Wiseman, and Luís Pina
(University of Illinois at Chicago, USA)
Property-based testing (PBT), introduced by Haskell’s Quickcheck, is becoming more popular with successful ports for other languages, such as Java’s junit-quickcheck. With PBT, developers write a property test and a data generator. The data generator takes a source of non-determinism and uses it to output well-formed data. The property test exercises the System Under Test (SUT) using the random well-formed data from the generator to ensure a particular property always holds (e.g., data serialized and deserialized should be equal to the original data). The PBT framework then performs many trials, each generating fresh data and executing the property test. A test failure shows a bug to developers, typically in edge-cases. A passing test gives some assurance on the quality of the SUT with regards to the property being tested. Unfortunately, well-known test coverage tools that are instrumental for understanding unit testing work poorly for PBT.
In this paper, we present PropCov, a tool for understanding statement coverage in PBT that also provides suggestions for coverage improvement. PropCov employs a novel combination of static analysis with PBT to approximate the maximum possible statement coverage, providing an effective measure of the PBT coverage and making suggestions to developers of where to improve existing tests. PropCov features an easily extensible architecture designed to support new languages, build systems, and PBT frameworks. We evaluated PropCov using 25 Java projects using junit-quickcheck or jqwik, totaling 293 properties, and found that existing tools report missed coverage that is impossible to reach (86% of lines that JaCoCo reports as not covered), which leads developers to consider hundreds of extra lines of code (2897). Unlike existing coverage tools, PropCov results are accurate — only 6.4% of all properties contain unfeasible code, and PropCov only misses 3% of reachable code. Using PropCov’s suggestions, we increased the coverage of 42 tests over 7 projects and found 5 new bugs in 4 projects.
Article Search Artifacts Available Article: issta26main-p344-p (type: Full Paper) doi:10.1145/3832128

Artifact for PropCov: Effective Coverage Reporting for Property-Based Testing (doi:10.5281/zenodo.21330969): This artifact accompanies the paper: PropCov: Effective Coverage Reporting for Property-Based Testing The artifact packages PropCov together with the scripts, configuration, benchmark inputs, and documentation needed to reproduce the experimental results described in the paper.

Uniting Bounded Verification and Transformer-Based Learning for Proactive IoT Security
Md Rashedul Hasan and Hamid Bagheri
(University of Nebraska-Lincoln, USA)
The proliferation of Internet-of-Things (IoT) ecosystems has introduced sophisticated interaction threats that emerge from unintended coordination between multiple applications, evading traditional single-app analysis. While formal verification provides soundness guarantees for interaction threats detection, it suffers from scalability limitations when analyzing large-scale deployments with hundreds of interacting applications. We present VeriWeave, a framework that synergistically combines bounded exhaustive formal verification, automated dynamic validation, and specialized transformer-based machine learning to detect interaction threats across application boundaries. VeriWeave employs static analysis to extract behavioral models from applications, uses bounded model checking to exhaustively enumerate potential interaction threat scenarios, at a static-derived scope obtained directly from the extracted models, and automatically validates these scenarios in instrumented environments to generate high-precision ground-truth labels. The validated results train specialized transformer models to predict exploitable interaction threats directly from code, achieving millisecond-scale predictions compared to several hours or days for formal analysis. Our evaluation on 3,732 real-world IoT applications together with a held-out benchmark of 4,000 additional applications (7,732 in total) demonstrates that VeriWeave reduces analysis time by 92% compared to pure formal methods while maintaining superior interaction threats detection accuracy. We validate cross-platform applicability on different ecosystems, confirming generalization. VeriWeave represents the first framework to bridge formal methods rigor with machine learning efficiency for interaction threats detection, enabling proactive vulnerability detection at scale through specialized models trained on validated interaction patterns.
Article Search Artifacts Available Article: issta26main-p346-p (type: Full Paper) doi:10.1145/3832129

Detecting IoT Interaction Threats with VeriWeave for Sustainable IoT Security (doi:10.6084/m9.figshare.30574706.v8): This artifact is the public reproduction package for VeriWeave, accompanying the ISSTA 2026 / PACMSE paper. It provides: (1) implementation and offline reproduction software for the VeriWeave interaction-threat detection pipeline (static analysis → Alloy-based bounded verification → dynamic validation → ...

Don’t Use a Cannon to Kill a Fly: Lightweight Model Editing for LLMs to Correct Deprecated API Recommendations
Guancheng Lin, Xiao Yu, Jacky Keung, Xing Hu, Xin Xia, and Alex X. Liu
(City University of Hong Kong, Hong Kong; Zhejiang University, China; Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security, China; Midea Group, China)
Pre-trained or fine-tuned on large code corpora, Large Language Models (LLMs) have demonstrated strong performance in code completion tasks. However, their embedded knowledge is constrained by the timeliness of training data, which often includes code using deprecated APIs. Consequently, LLMs frequently generate deprecated APIs that will no longer be supported in future versions of third-party libraries. While retraining LLMs on updated codebases could refresh their API knowledge, this approach is computationally expensive. Recently, lightweight model editing methods have emerged to efficiently correct specific knowledge in LLMs. However, it remains unclear whether these methods can effectively update deprecated API knowledge and enable edited models to generate up-to-date APIs. To address this gap, we conduct the first systematic study applying 10 state-of-the-art model editing techniques to update deprecated API knowledge in three LLMs: Qwen2.5-Coder, CodeGemma, and DeepSeek-Coder. We introduce EDAPIBench, a dedicated benchmark featuring over 70 deprecated APIs from 8 popular Python libraries, with more than 3,000 editing instances. Our results show that the parameter-efficient fine-tuning method AdaLoRA achieves the best performance in enabling edited models to generate correct, up-to-date APIs, but falls short in Specificity (i.e., the editing influences untargeted knowledge). To resolve this, we propose AdaLoRA-L, which defines "Common API Layers" (layers within the LLMs with high importance across all APIs, storing general knowledge and excluded from editing) and restricts edits exclusively to "Specific API Layers" (layers with high importance only for the target API, storing the API-specific knowledge). Experimental results demonstrate that AdaLoRA-L significantly improves Specificity while maintaining comparable performance across other evaluation metrics.
Article Search Article: issta26main-p348-p (type: Full Paper) doi:10.1145/3832130

ScratchNet: A Multi-modal Benchmark for Evaluating and Advancing LLMs on Scratch Programming Tasks
Yuan Si, Simeng Han, Daming Li, Hanyuan Shi, and Jialu Zhang
(University of Waterloo, Canada; Stanford University, USA; Independent Researcher, USA; Independent Researcher, China)
Large language models (LLMs) have achieved impressive performance on text-based programming tasks, yet they remain unreliable for block-based languages such as Scratch. Scratch programs feature deeply nested, nonlinear structures, event-driven concurrency across multiple sprites, and tight coupling between code and multimedia assets---properties that differ fundamentally from textual code. Consequently, LLMs frequently misinterpret Scratch semantics and propose large, invasive edits that are syntactically valid but semantically misaligned when repairing buggy programs.
We introduce ScratchNet, the first executable benchmark designed to systematically evaluate and advance LLM-based repair for Scratch programs. The benchmark comprises 100 carefully curated projects from the public Scratch repository, each selected for high structural and semantic complexity. Every project is paired with an executable test suite, a bug description and corresponding fix, block-level edit constraints that define a minimal semantically correct repair, and the multimedia assets required for faithful execution. We construct the benchmark through a human-in-the-loop pipeline that combines automated project mining with expert validation of trigger--mechanism--outcome semantics and representative bug patterns, with particular emphasis on event ordering, concurrency, and state management.
To enable rigorous and reproducible evaluation, we propose a three-layer executable protocol that measures functional correctness through VM-level execution, repair quality through block-level edit distance and behavioral trajectory comparisons, and explanation quality through structured rubrics. Using this benchmark, we study project and bug understanding, trigger and mechanism identification, functional repair, and the effect of lightweight domain adaptation. ScratchNet establishes a reproducible foundation and a closed-loop framework for evaluating and post-training LLMs on block-based programming tasks.
Article Search Article: issta26main-p356-p (type: Full Paper) doi:10.1145/3832131

MG-Fuzz: Model-Guided Fuzzing for Unsafe Scenario Discovery in Autonomous Driving Systems
Yulong Lyu, Ruiqi Hong, Jiawan Wang, Jun Sun, and Lei Bu
(Nanjing University, China; Singapore Management University, Singapore)
As autonomous driving systems (ADS) are increasingly deployed in real-world environments, discovering diverse unsafe driving scenarios remains a fundamental yet difficult problem. Existing scenario generation and testing approaches often rely on black-box exploration or externally-observed heuristic feedback, which struggle to effectively guide the search toward high-risk scenarios induced by complex decision-making behaviors. A key difficulty stems from the fact that unsafe behaviors in ADS often arise from internal decision-making logic, which can induce structured and discontinuous responses that are hard to effectively explore using purely black-box guidance. Consequently, current tools tend to repeatedly discover a narrow set of similar unsafe scenario types, limiting their ability to expose diverse and previously unseen failure modes.
In this paper, we propose MG-Fuzz, a model-guided, multi-objective fuzzing framework for unsafe scenario discovery in autonomous driving systems. Our approach extracts an automaton model that captures the core control logic of the ADS decision-making component, and leverages this model as structured guidance for search-based scenario exploration. To systematically drive the exploration process, MG-Fuzz integrates model-based metrics derived from the automaton with complementary safety metrics, enabling effective evaluation and prioritization of generated driving scenarios across diverse unsafe behavior types. MG-Fuzz has been developed and thoroughly evaluated through extensive experiments on autonomous driving systems. Experimental evidence indicates that MG-Fuzz successfully detects 18 distinct types of unsafe driving scenarios, marking a substantial improvement in detection breadth relative to current state-of-the-art tools.
Article Search Article: issta26main-p364-p (type: Full Paper) doi:10.1145/3832132

Mind the Gap: An Empirical Study of Synchronization Gaps, Delays, and Missed Opportunities in Software Forks
Jiaying Zhu, Lyuye Zhang, Jiahui Wu, Chengyue Liu, and Yang Liu
(Nanyang Technological University, Singapore; Nankai University, China)
Fork-based development enables parallel evolution of software, but unsynchronized contributions create persistent divergence: security patches, bug fixes, and quality improvements often fail to propagate across fork families, leaving downstream users exposed to known vulnerabilities or bugs and missing massive opportunities to improve the other repositories in the family. We present the first large-scale empirical study of fork synchronization, analyzing popular GitHub fork families with 3,820 actively maintained forks, and developed a monitoring platform to mine the valuable commits and promote their swift merging.
Our findings reveal a synchronization paradox: while 90% of submitted pull requests are merged, only 6.92% of fork commits ever appear in PRs, leaving massive fork development permanently unsynchronized across the families. Synchronization delay is pervasive and structurally uneven where fork propagation accounts for 72.9% of end-to-end commit lifecycle delay. Contrary to common assumptions, PR rejection is rarely caused by technical incorrectness; instead, 65% of rejections stem from superseded contributions, process violations, or maintainer policy decisions.
Based on these insights, we develop a three-stage syncability assessment pipeline that identifies fork-local commits that are both sync-worthy (broadly beneficial) and sync-eligible (technically and policy-compatibly portable). Applied to 0.5 million fork-local commits, our pipeline surfaces 12,284 sync-ready commit--repository pairs, demonstrating that our approach identifies practically valuable changes. To further validate the security impact, we manually reviewed 153 security-related commit--repository pairs and confirmed 83 as potential 1-day vulnerabilities, for which we produced 35 proof-of-concept of exploit demonstrations and filed issues to the affected repositories. Our monitoring platform enables continuous, near-real-time detection of synchronization opportunities across fork families, improving the sustainability of fork-based open-source ecosystems.
Article Search Article: issta26main-p366-p (type: Full Paper) doi:10.1145/3832133

PoCE: Automated Proof-of-Concept Synthesis using Large Language Models for Robust Validation
Tanusree Das Tithy, Lamia Hasan Rodoshi, Ayman Rafid Azahar, Amlan Abhidarshi, Tabassum Faruk, Fahmid Al Rifat, and Faysal Hossain Shezan
(University of Texas at Arlington, USA; University of Texas at Austin, USA)
Vulnerability reports play a critical role in software repair, with Proof-of-Concept (PoC) tests serving as one of their most essential components. PoC tests enable software developers to reliably reproduce reported vulnerabilities and subsequently deploy patches. However, generating effective PoCs is costly, expertise-intensive, and increasingly challenging due to the diversity of modern software ecosystems and their complex dependencies. Inadequate or incorrect PoCs can significantly delay patch deployment, thereby increasing the window of exposure to attacks. Prior work on automated PoC generation struggles to produce comprehensive and reliable testing. In this work, we present an automated PoC generation framework, PoCE, capable of generating PoCs across diverse software systems by handling varied input formats and complex execution contexts using large language models. PoCE integrates structured in-context learning, retrieval-augmented generation, and iterative chain-of-thought reasoning to expand an initial successful PoC into multiple validated variants. These variants are executed in controlled environments to confirm success. We evaluate PoCE on thirteen widely used software projects, including TensorFlow, Yasm, Zlib, Liblouis, Cflow, Pytorch, Node.js, TCPDUMP, Fig2dev, Binutils, libsndfile, LibTIFF, and libsixel. Our approach achieves a success rate of 77.7% and generates multiple PoC variants for the most vulnerable cases, uncovering alternative trigger paths and edge conditions. We discover 68 zero-day PoCs and identify 26 previously unknown zero-day vulnerabilities in cross-layer software.
Article Search Artifacts Available Article: issta26main-p367-p (type: Full Paper) doi:10.1145/3832134

PoCE (doi:10.5281/zenodo.21330504): Research Project | Proof-of-concept Exploit Generation with LLMs This repository contains tools for generating proof-of-concept exploits for different software using LLMs, as well as their associated testing data in JSON format. Prepare and Run the following Steps: First install ollama by following the ...

Branch-Level Fault Localization in ADS Planning via Temporal Coverage Analysis
Sangmin Woo, Dohyun Kim, Donghwan Shin, and Yongdae Kim
(KAIST, Republic of Korea; University of Sheffield, UK)
Planning failures in Automated Driving Systems (ADS) are increasingly detected through simulation-based testing, yet localizing their root causes within planning code remains a major challenge. Planning modules execute complex rule-based decision logic over hundreds of frames in a closed-loop interaction with the environment, where faults trigger observable failures only after temporal gaps and under specific execution contexts. These characteristics make traditional spectrum-based fault localization ineffective, as faulty behavior is obscured by execution-level coverage aggregation and limited test diversity. In this paper, we study the problem of debugging planning failures and present a temporal coverage analysis approach for localizing faults in rule-based planning modules. Our key insight is that, while execution-aggregated coverage masks fault behavior, frame-level execution dynamics reveal distinctive temporal signatures that indicate when and how faulty branches activate. Leveraging this insight, our approach first identifies a suspicious frame using planning semantics, and then ranks candidate branches by analyzing their execution behavior within a localized temporal window. We evaluate our approach on 221 reproducible non-collision Apollo planning failures, covering immobility and emergency mission failures. Our results show that temporal coverage analysis enables accurate suspicious-frame identification and substantially reduces branch inspection effort compared to oracle-based and random baselines, effectively localizing faults from a single failing execution. We further analyze failure cases that lack observable execution signals to clarify the fundamental limits of execution-based localization. Overall, this work demonstrates that temporal execution analysis provides a practical and effective foundation for debugging planning failures in rule-based ADS planning modules.
Article Search Article: issta26main-p368-p (type: Full Paper) doi:10.1145/3832135

Better Call Grep: Evaluating and Improving Grep-Like Lexical Retrieval for Repository-Level Code Completion
Baoyi Wang, Xingliang Wang, Guochang Li, Chen Zhi, Junxiao Han, Xinkui Zhao, Nan Wang, Shuiguang Deng, and Jianwei Yin
(Zhejiang University, China; Hangzhou City University, China; Shenzhou Aerospace Software Technology, China)
Repository-level code completion remains challenging for large language models (LLMs), as it requires reasoning over cross-file dependencies while under limited context windows. To address this challenge, prior work has adopted Retrieval-Augmented Generation (RAG) frameworks based on semantic indexing or structure-aware graph analysis. Although effective, these approaches introduce substantial computational overhead for index construction and maintenance, which hinders their practicality in real-world development. Motivated by common developer workflows that rely on lightweight search utilities (e.g., ripgrep) to locate relevant code, we revisit a fundamental yet underexplored question: how far can simple, index-free lexical retrieval go in supporting repository-level code completion before more complex retrieval mechanisms become necessary? To answer this question, we systematically explore the potential of lightweight, index-free, intent-aware lexical retrieval through extensive empirical analysis. We first introduce Naive GrepRAG, a baseline framework where LLMs autonomously generate ripgrep commands to localize relevant context. Our preliminary experiments show that even this basic implementation achieves performance comparable to sophisticated graph-based baselines. Further analysis reveals that its effectiveness stems from retrieving code fragments that are lexically precise and spatially closer to the completion site. However, we identify key limitations of this approach, including sensitivity to noisy matches caused by high-frequency ambiguous keywords and context fragmentation due to rigid truncation boundaries. To address these issues, we propose GrepRAG, which augments lexical retrieval with a lightweight post-processing pipeline featuring identifier-weighted re-ranking and structure-aware deduplication. Extensive evaluation on CrossCodeEval and RepoEval_Updated demonstrates that GrepRAG consistently outperforms state-of-the-art (SOTA) methods. In particular, on CrossCodeEval, GrepRAG achieves 7.04–15.58% relative improvement in code exact match (EM) over the best baseline.
Article Search Article: issta26main-p388-p (type: Full Paper) doi:10.1145/3832136

Beyond the Surface: Towards Feature-Driven Fuzzing on the Chrome Browser
Chaoyuan Peng, Muhui Jiang, Yajin Zhou, and Lei Wu
(Zhejiang University, China; BlockSec, China; Chinese University of Hong Kong, Hong Kong)
The Chrome browser constitutes a complex software system responsible for processing and rendering diverse web content. Despite extensive testing and security measures implemented by the vendor and the community, the inherent complexity of this system makes the complete elimination of vulnerabilities practically infeasible. Existing DOM and API fuzzing techniques inadequately address the expanded attack surface introduced by Chrome features and extensions, resulting in a substantial number of elusive vulnerabilities remaining undetected. This paper presents Feazzer, an efficient feature-driven Chrome browser fuzzing framework designed to detect elusive vulnerabilities introduced by Chrome features. Our approach leverages hybrid programs comprising HTML and Chrome extensions with systematically clustered feature options to explore deep browser states in Chrome that existing fuzzers fail to reach. We introduce a message-guided fuzzing mechanism that reduces feature conflicts and enhances the semantic quality of generated test cases. Our comprehensive evaluation across multiple Chrome versions demonstrates that Feazzer achieves up to 231.1% improvement in code coverage compared to state-of-the-art fuzzers. Feazzer has discovered 39 previously unknown bugs in Chrome, with 6 assigned CVEs and acknowledgment of over $55,000 in bug bounties from the vendor. Notably, 2 bugs are rated as critical and 27 as high severity, demonstrating the effectiveness of Feazzer in discovering impactful bugs.
Article Search Article: issta26main-p403-p (type: Full Paper) doi:10.1145/3832137

SemantiX: A Compatibility Checker between Applications and Compositions of Distributed Systems
Yifei Sun and Ji-Yong Shin
(Northeastern University, USA)
Modern distributed applications compose multiple services with different consistency guarantees. Mismatches between application requirements and chosen system semantics can introduce subtle bugs, but verifying compatibility across complex applications is both challenging and time-consuming. In particular, existing testing approaches or rigorous formal verification lacks flexibility and cannot promptly provide correctness guarantees on multi-semantic compositions.
We present SemantiX, a framework for checking semantic compatibility between applications and compositions of distributed systems with heterogeneous consistency models. SemantiX embeds formally defined distributed system modules of consistency semantics which include the first formal definition of visibility constraints. SemantiX introduces the AppGraph approach for systematically modeling complex applications. Applications modeled using AppGraphs can be checked for their compatibility against a combination of distributed system modules that the user is considering. Alternatively, SemantiX can search for combinations of modules compatible with the application. We present three case studies on a movie streaming service, an e-commerce platform, and a cross-service causal distributed storage service. We demonstrate that SemantiX can capture complex service compositions with minimal encoding effort and quickly check compositional compatibility with underlying systems or find the compatible system configurations. SemantiX enables developers to efficiently verify distributed application designs and explore alternative consistency configurations, supporting more agile development.
Article Search Artifacts Available Article: issta26main-p405-p (type: Full Paper) doi:10.1145/3832138

SemantiX: A Compatibility Checker between Applications and Compositions of Distributed Systems (Artifact) (doi:10.5281/zenodo.21509647): The Zenodo archive contains the source code, QEMU disk image with dependencies preinstalled, and instructions.

On the Role of Large Language Models in Robustness-Guided Requirement Falsification
Ali Kaya and Ivan Porres
(Åbo Akademi University, Finland)
Robustness-guided falsification supports the validation of cyber-physical systems by searching for system traces that violate formal temporal requirements. Existing falsification methods typically treat candidate generation as a numerical search over the input space, guided by quantitative robustness feedback. This paper studies a different mechanism: a guarded advisory proposer that uses a large language model (LLM) to generate candidate inputs. The LLM-based proposer suggests candidate inputs using the following information: the requirement to validate, the interface of the system under test (SUT), and the history of prior candidate evaluations. Admission gates ensure that only valid candidate inputs are considered for execution. We evaluate this approach on 19 falsification requirements from the ARCH-COMP 2024 CPS benchmark and on five synthetic problems. The results show that LLM-based proposing is sensitive to model choice and context representation. Among the tested variants, GPT-P, which uses a GPT model with the plain context representation, is the most reliable representative variant. On ARCH-COMP, GPT-P reaches perfect falsification on 16 of 19 requirements, obtains at least one falsification on all 19 requirements, and, on 8 requirements, uses the fewest executions among the reported tools that also achieve perfect falsification. However, its performance is heterogeneous. We explain this behavior through three classes of falsification problems: informative context cues, ambiguous context cues with a strong robustness gradient, and ambiguous context cues with a weak robustness gradient. Synthetic benchmark problems reproduce these classes under controlled mechanisms. Overall, GPT-P performs best on the informative-context-cue class, where the requirement and SUT interface expose useful candidate hypotheses that the LLM can exploit before extensive robustness-feedback search is needed. To our knowledge, existing robustness-guided falsification methods do not exploit such semantic context cues when proposing candidate inputs.
Article Search Artifacts Available Article: issta26main-p411-p (type: Full Paper) doi:10.1145/3832139

Artifact Evaluation Package for "On the Role of Large Language Models in Robustness-Guided Requirement Falsification" (doi:10.5281/zenodo.21294060): This artifact evaluation package supports the paper “On the Role of Large Language Models in Robustness-Guided Requirement Falsification", accepted for presentation in ISSTA 2026. It packages the paper-specific Guarded LLM proposer, GLLMPro, archived experimental data, deterministic result-generation scripts, and a ...

Secrets Unlocked: Evaluating LLMs for Secrets Detection in Android Apps
Marco Alecci, Jordan Samhi, Tegawendé F. Bissyandé, and Jacques Klein
(University of Luxembourg, Luxembourg)
Mobile apps frequently embed sensitive secrets, such as API keys, access tokens, client secrets, and private keys that support internal functionality or enable integration with external systems and third-party services. Developers frequently embed these secrets into Android apps, which allows attackers to extract them through reverse engineering. Once exposed, attackers can exploit them to access sensitive data, manipulate resources, or abuse APIs, resulting in severe security and potential financial risks. In this paper, we present the first large-scale empirical evidence that off-the-shelf large language models (LLMs) can automatically identify secrets in Android apps without any domain-specific prior knowledge, thereby substantially lowering the barrier for attackers. On a benchmark of 5135 Android apps from prior work, LLMs rediscovered 93% of previously known secrets and identified 4361 additional valid credentials (+195%). Extending our analysis to 50 000 Google Play apps collected between August and October 2025, we conducted the largest-scale study to date on secret detection in Android apps, identifying secrets in 17 590 apps (35%). Among the 18 908 detected secrets, 1802 remained active at discovery, including, among others, critical credentials such as Stripe payment keys, OpenAI API keys, and GitHub personal access tokens. We responsibly contacted all the affected developers, of whom 170 confirmed the issues and updated their apps accordingly. Our findings empirically demonstrate the reality of vibe hacking: anyone can now leverage publicly accessible AI models to perform complex offensive security tasks with minimal expertise.
Article Search Article: issta26main-p419-p (type: Full Paper) doi:10.1145/3832140

SAGE: Signal-Amplified Guided Embeddings for Vulnerability Detection
Zhengyang Shan, Xu Qian, Jiayun Xin, Minghui Xu, Yue Zhang, Zhen Yang, Hao Wu, and Xiuzhen Cheng
(Shandong University, China; Quancheng Laboratory, China; Nanjing University, China)
Software vulnerabilities are a primary threat to modern infrastructure. While static analysis and Graph Neural Networks have long served as the foundation for vulnerability detection, the emergence of Large Language Models (LLMs) has introduced a transformative paradigm driven by superior semantic reasoning and cross-environment generalization. However, in the context of LLM-based vulnerability detection, we identify a fundamental bottleneck in these models termed Signal Submersion: a state where features related to vulnerability are activated internally but numerically overwhelmed by dominant functional semantics. To address this, we propose SAGE (Signal-Amplified Guided Embeddings), a framework that shifts from passive signal submersion to active signal recovery. SAGE integrates task-conditional Sparse Autoencoders (SAEs) to isolate and amplify these faint vulnerability signals. Extensive evaluations on BigVul, PrimeVul, and PreciseBugs demonstrate that SAGE achieves state-of-the-art performance. Notably, SAGE mitigates Signal Submersion by increasing the internal Signal-to-Noise Ratio (SNR) by 12.7× via sparse manifold projection. This mechanistic intervention enables a 7B model to achieve up to 318% Matthews Correlation Coefficient (MCC) gains on unseen distributions and a 319% gain on classic datasets. By maintaining robust performance across 13 programming languages and outperforming 34B baselines, SAGE establishes a more efficient and scalable path to software security than simple parameter scaling.
Article Search Article: issta26main-p425-p (type: Full Paper) doi:10.1145/3832141

LogicHunter: Testing LLM Agent Frameworks with an Agentic Oracle
Minghui Long, Yanjie Zhao, and Haoyu Wang
(Huazhong University of Science and Technology, China)
Large Language Model (LLM) agent frameworks such as LangChain, LlamaIndex, and CrewAI have become critical infrastructure powering production AI systems, yet they remain severely under-tested due to fundamental challenges in automated testing. Unlike traditional software, where crashes serve as reliable oracles, defects in these pure Python frameworks manifest as ordinary exceptions or silent semantic failures, creating profound oracle ambiguity. This problem is exacerbated by strict type governance through Pydantic schemas and complex protocol requirements that cause existing fuzzers to generate overwhelming invalid inputs, while traditional test generators produce only trivial cases with weak regression assertions.
We present LogicHunter, a fuzzing framework that addresses both the generation and oracle challenges through active specification-aware testing. LogicHunter employs specification-driven generation that systematically fuses formal type constraints with authentic usage patterns from real-world repositories, synthesizing inputs that are valid by construction yet semantically extreme, equipped with behavioral probes to expose silent failures. To resolve oracle ambiguity, we introduce the Agentic Oracle, which transcends passive classification by actively retrieving documentation, navigating source code, and inspecting runtime states through a ReAct-based architecture with Dual-Layer State Management and Dual-Stream Memory. Evaluated on three widely deployed frameworks, LogicHunter discovered 40 previously unknown bugs with 30 confirmed and 26 fixed by developers, while state-of-the-art baselines reported no bugs as final findings. The Agentic Oracle achieves 91.17% precision, surpassing the best passive approach at 29.27% by 61 percentage points.
Article Search Artifacts Available Article: issta26main-p434-p (type: Full Paper) doi:10.1145/3832142

LogicHunter: Artifact for "Testing LLM Agent Frameworks with an Agentic Oracle" (doi:10.5281/zenodo.21936443): LogicHunter is a testing framework for LLM agent libraries such as LangChain, LlamaIndex, and CrewAI. This artifact contains the source code, API lists, usage artifacts, evaluation utility scripts, and documentation for LogicHunter version 1.0.0 accompanying the ISSTA 2026 paper "LogicHunter: Testing LLM Agent ...

A Temporal Reasoning Benchmarking Framework for LRMs via Difficulty-Controlled and Dynamic Test Generation
Shide Zhou, Kailong Wang, Ling Shi, and Haoyu Wang
(Huazhong University of Science and Technology, China; National University of Singapore, Singapore; Nanyang Technological University, Singapore)
Defining the reasoning boundaries and ensuring the reliability of Large Reasoning Models (LRMs) remains a critical challenge. Current benchmarks primarily rely on static datasets susceptible to data contamination or synthetic tasks lacking fine-grained difficulty control. Furthermore, standard outcome-based evaluations often conceal reasoning flaws by neglecting the reasoning process.
To address these limitations, we introduce TRACE, a testing framework that models temporal reasoning as constraint satisfaction problems via Allen’s Interval Algebra. This approach enables precise regulation of logical complexity and incorporates a Trace-Based Verification Oracle to validate reasoning faithfulness. Using this framework, we construct TRACEBench, an extensive benchmark comprising 1,200 synthesized test instances across graded difficulty levels. We employ TRACE to evaluate eight widely used LRMs on TRACEBench. The results confirm a strong negative correlation between model performance and our difficulty metric (Pearson’s r ≈ −0.96), validating the effectiveness of our difficulty control mechanism. Moreover, our trace-based analysis exposes significant discrepancies between reasoning validity and final answers, revealing a high spurious guessing rate of approximately 28% in mid-sized models. In addition, we diagnose scale-dependent failure modes, ranging from Degenerative Loops in small models to Reasoning Explosion in advanced architectures. TRACE thus provides a robust, automated platform for benchmarking the true temporal reasoning capabilities of LRMs.
Article Search Article: issta26main-p437-p (type: Full Paper) doi:10.1145/3832143

A Comprehensive Empirical Analysis of Patch Presence Testing: Capabilities, Limitations, and Paths Forward
Xiaobei Zhang, Yaowen Zheng, Wu Luo, Shijun Zhao, Yongsheng Tao, Dan Meng, and Rui Hou
(Institute of Information Engineering at Chinese Academy of Sciences, China; School of Cyber Security at University of Chinese Academy of Sciences, Beijing, China)
Patch presence testing determines whether a binary incorporates the security fixes associated with a known vulnerability and has become increasingly important in software supply-chain security. However, despite numerous proposed techniques, the field still lacks a comprehensive understanding of the practical capabilities and limitations of existing approaches. Fundamental questions remain unanswered, including how well current tools perform in real-world settings, which vulnerability or patch characteristics shape detection accuracy, and what underlying factors limit the effectiveness of all existing tools.
To address these issues, we conduct the first systematic and in-depth empirical study of patch presence testing for C/C++ binaries. We construct a high-fidelity benchmark comprising 561 CVEs across ten widely used projects, with binaries compiled under diverse configurations. Using this dataset, we perform an extensive evaluation of five state-of-the-art tools representing both syntactic and semantic methodologies. Our findings show that: (1) accuracy reported in prior work reflects only cases where tools successfully generate outputs, whereas in practice many tools frequently fail to produce any result; (2) patch semantics, code scale, and compiler options exert a strong influence on accuracy, whereas CWE categories provide little predictive value; (3) common failures fall into two major categories: algorithmic limitations, such as the inability to detect subtle or evolved patches, and engineering deficiencies, such as failures triggered by function-level structural modifications or symbol duplication. Building on these findings, we develop two improvement strategies and integrate them into state-of-the-art tools, resulting in notable gains in both accuracy and overall reliability for patch detection.
Article Search Artifacts Available Article: issta26main-p441-p (type: Full Paper) doi:10.1145/3832144

Dataset and Results for "A Comprehensive Empirical Analysis of Patch Presence Testing" (doi:10.5281/zenodo.21287501): Data artifact and replication package for the ISSTA 2026 paper "A Comprehensive Empirical Analysis of Patch Presence Testing: Capabilities, Limitations, and Paths Forward." This repository snapshot contains the study's dataset metadata (562 test-set CVE entries and 559 reference entries across 10 open-source projects: ...

An Empirical Study of Speculative Decoding on Software Engineering Tasks
Yijia Li, Junkai Chen, Xing Hu, and Xin Xia
(Zhejiang University, China; Singapore Management University, Singapore; Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security, China)
Large Language Models (LLMs) have become widely used for Software Engineering (SE) tasks, spanning from function-level code generation to complex repository-level workflows. However, the high latency of autoregressive inference remains a significant bottleneck, hindering their deployment in interactive environments. While Speculative Decoding (SD) offers a promising technique for lossless acceleration, prior research on long-context repository-level tasks and complex agentic interactions remains limited. To bridge this gap, we present a systematic empirical study to evaluate the effectiveness of SD in SE tasks. We benchmark a representative spectrum of strategies, encompassing both model-based and model-free methods, across generation, editing, and repair scenarios. Our empirical results show that SD provides clear acceleration potential for SE tasks, but its realized benefits jointly vary with model architecture and task scenario. Specifically, model-based approaches are well-suited for code generation, whereas model-free methods are better adapted to repository-level repair and editing scenarios. We further observe that the repetitiveness of SE tasks improves the performance of model-free methods, while complex agentic workflows can introduce repetitive failure modes that skew acceleration measurements. In contrast to natural language tasks, the higher predictability of SE tasks allows for more aggressive hyperparameter settings. Our findings provide practical guidance for selecting, configuring, and evaluating SD methods in SE scenarios.
Article Search Article: issta26main-p452-p (type: Full Paper) doi:10.1145/3832145

Lookahead-Then-Verify: Reliable Constrained Decoding for Diffusion LLMs under Context-Free Grammars
Yitong Zhang, Yongmin Li, Yuetong Liu, Jia Li, Xiaoran Jia, Zherui Li, and Ge Li
(Beihang University, China; Peking University, China; Tsinghua University, China; Beijing Institute of Technology, China; Nanyang Technological University, Singapore)
Diffusion Large Language Models (dLLMs) have demonstrated promising capabilities and are increasingly used to produce formal languages defined by context-free grammars, such as source code and chemical expressions. However, as probabilistic models, they still struggle to generate syntactically valid outputs reliably. A natural and promising direction to address this issue is to adapt constrained decoding techniques to enforce grammatical correctness during generation. However, applying these techniques faces two primary obstacles. On the one hand, the non-autoregressive nature of dLLMs renders most existing constrained decoding approaches inapplicable. On the other hand, current approaches specifically designed for dLLMs may allow intermediate outputs that are impossible to complete into valid sentences, which significantly limits their reliability in practice.
To address these challenges, we present LAVE, a constrained decoding approach specifically designed for dLLMs. Our approach leverages a key property of dLLMs, namely their ability to predict token distributions for all positions in parallel during each forward pass. Whenever a new token is proposed by the model, LAVE performs lookahead using these distributions to efficiently and reliably verify the validity of the proposed token. This design enforces reliable constraints by preserving the potential for intermediate outputs to be extended into valid sentences. Extensive experiments across four widely used dLLMs and five representative benchmarks demonstrate that LAVE consistently outperforms existing baselines and achieves improvements in syntactic correctness, while incurring negligible runtime overhead.
Article Search Article: issta26main-p456-p (type: Full Paper) doi:10.1145/3832146

ChromaEyes: Detecting Inconsistencies of User Interface Elements between Light and Dark Modes of Web Applications
K C Shweta, Byungchul Tak, Tegawendé F. Bissyandé, and Dongsun Kim
(Korea University, Republic of Korea; Kyungpook National University, Republic of Korea; University of Luxembourg, Luxembourg)
Dark mode interfaces in web applications have gained widespread adoption because of improved user comfort and reduced power consumption. While these interfaces can be implemented through built-in support or browser extensions that convert light mode layouts, inconsistencies frequently arise during the conversion process, including invisible UI elements, misplaced components, and incorrect color mappings. Despite the prevalence of these issues, no existing approaches systematically detect such inconsistencies between light and dark mode interfaces. Our preliminary study shows that popular commercial vision language models and accessibility issue detectors are ineffective for this task.
This paper presents ChromaEyes, a novel approach to automatically detecting inconsistencies of graphical user interface elements between light and dark mode layouts of web applications. Detecting such inconsistencies is inherently challenging given that, since mode conversion intentionally changes colors and contrast, UI elements in light and dark modes are expected to look different. Thus, pixel-wise or visual comparison cannot distinguish intentional adaptations from actual errors. ChromaEyes addresses this challenge by analyzing semantic roles and functional meanings of UI elements, enabling accurate correspondence detection between visually distinct but functionally equivalent components. We evaluate our approach on 2,009 screenshot pairs captured from 196 real web applications (147 with native dark mode support and 49 with browser extension-based conversion). ChromaEyes achieves 96.19% accuracy at the screenshot level and 97.95% at the application level, significantly outperforming vision-language models (e.g., GPT-4o) and state-of-the-art accessibility issue detectors (e.g., OwlEye, axe DevTools).
Article Search Artifacts Available Article: issta26main-p459-p (type: Full Paper) doi:10.1145/3832147

Embracing the Dark Side: Detecting and Repairing Inconsistencies between Light and Dark Modes of Web Applications (doi:10.5281/zenodo.21715790): The dataset consists of light and dark mode screenshots in screen-size "iPhone 12 Pro" along with UPstage OCR detection and JSON file, UIED GUI element detection and JSON file. This collection of datasets is used to identify the GUI inconsistency between light and dark modes.

Environmental Injection Attacks against GUI Agents in Realistic Dynamic Environments
Yitong Zhang, Ximo Li, Liyi Cai, and Jia Li
(Beihang University, China; Tsinghua University, China; Peking University, China)
Graphical User Interface (GUI) agents are increasingly deployed to interact with online web services, yet their exposure to open-world content renders them vulnerable to Environmental Injection Attacks (EIAs). In these attacks, an attacker can inject crafted triggers into a website to manipulate the behavior of other users’ GUI agents. In this paper, we find that most existing EIA studies fall short of realism. In particular, they fail to capture the dynamic nature of real-world websites, often assuming that a trigger’s on-screen position and surrounding visual context remain largely consistent between training and testing. To better reflect practice, we introduce a realistic dynamic-environment threat model in which the attacker is a regular user and the trigger is embedded within a dynamically changing environment. Under this threat model, existing approaches largely fail, suggesting that their effectiveness in exposing GUI agent vulnerabilities has been overestimated.
To expose the hidden vulnerabilities of existing GUI agents effectively, we propose Chameleon, an attack framework with two key components designed for dynamic environments. (1) To synthesize more realistic training data, we introduce LLM-Driven Environment Simulation, which automatically generates diverse, high-fidelity webpage simulations that mimic the variability of real-world dynamic environments. (2) To optimize the trigger more effectively, we introduce Attention Black Hole, which converts attention weights into explicit supervisory signals. We evaluate Chameleon on six realistic websites and four representative LVLM-powered GUI agents. Across these settings, it significantly outperforms existing methods. Ablation studies confirm that both components are critical to performance, and a closed-loop sandbox experiment further demonstrates that Chameleon can successfully hijack agent behavior in conditions that closely mirror real-world usage. Our results uncover a critical, previously underexplored vulnerability of GUI agents in realistic dynamic environments and establish a robust foundation for future research on defenses for open-world GUI agent systems.
Article Search Article: issta26main-p473-p (type: Full Paper) doi:10.1145/3832148

AttnCompress: Dynamic Attention-Guided Trajectory Compression for Software Engineering Agents
Zhengran Zeng, Yixin Li, Rui Xie, Wei Ye, and Shikun Zhang
(Peking University, China)
The transition from human-centric assistance to Autonomous Software Engineering (ASE) agents has enabled the resolution of complex real-world SE tasks. However, the trial-and-error nature of these agents generates lengthy interaction trajectories, creating severe bottlenecks in terms of context window limits and cost. While context compression offers a potential remedy, prior approaches suffer from static pruning strategies and granularity mismatches, often failing to preserve the semantic dependencies and syntactic details crucial for SE tasks. To strictly preserve critical task evidence while reducing context length, we introduce AttnCompress, a dynamic attention-guided trajectory compression framework. Unlike existing approaches, AttnCompress bridges the gap between semantic integrity and dynamic adaptability through three key mechanisms: (1) structure-aware segmentation via perplexity (PPL) spikes to preserve the syntactic structure of code and logs; (2) relevance estimation using proxy attention weights to quantify the precise relevance of historical blocks to the agent’s current reasoning; and (3) a dynamic rolling window to re-evaluate and recall historical context as the task evolves. Extensive evaluation on SWE-Bench-Verified and Multi-SWE-Bench demonstrates that AttnCompress achieves a pass rate of 53.17%, outperforming prior state-of-the-art baselines while reducing token consumption by 21.6% and total costs by 33.6%. The framework proves to be model-agnostic and generalizes effectively across diverse programming languages.
Article Search Article: issta26main-p483-p (type: Full Paper) doi:10.1145/3832149

From Draft to Precision: Iterative Agentic Framework for Intent-Aware Code Summarization
Yifei Ge, Chunrong Fang, Zhenyu Chen, and Juan Zhai
(Nanjing University, China; University of Massachusetts at Amherst, USA)
Code summarization aims to generate natural language (NL) descriptions for code snippets to assist developers in understanding and maintaining programs. Recent advances in large language models (LLMs) have substantially improved the quality of automatic code summarization. In real-world projects, code comments are written to serve different developer intents, which has motivated the development of intent-aware summarization methods. However, existing intent-aware methods still struggle to fully capture developer intent, often producing incomplete or misaligned summaries. Our empirical study reveals that real-world comments are typically refined through multiple rounds of developer revision, which we believe is key to achieving high quality. However, current methods typically overlook this iterative refinement. To bridge this gap, we propose a plan-conditioned revision framework that improves code summarization through iterative revision. It integrates two collaborating agents: a Generator that produces and revises summaries, and a Reviewer that assesses the generated summary and drafts targeted revision plans to guide subsequent edits using available contextual information. By iteratively refining summaries with revision plans and contextual information, our framework yields summaries that are more accurate, complete, and better aligned with developer intent. Extensive experiments on an intent-annotated CSN-Java benchmark demonstrate the effectiveness of our method. On automatic metrics, our method achieves a 28.75% improvement over the state-of-the-art baseline methods across all intents. Besides, in human studies, our framework yields average gains of over 20% in usefulness, adequacy, and intent alignment. It also achieves the highest Top-1 preference (over 40% across all intents), underscoring its practical advantage for developers
Article Search Artifacts Available Article: issta26main-p506-p (type: Full Paper) doi:10.1145/3832150

Artifact for From Draft to Precision: Iterative Agentic Framework for Intent-Aware Code Summarization (doi:10.5281/zenodo.21886264): Archived research artifact accompanying the ISSTA 2026 paper "From Draft to Precision: Iterative Agentic Framework for Intent-Aware Code Summarization". The release contains the core Generator--Reviewer framework implementation, prompt templates, voting-based intent-classification wrapper, and supporting utilities. ...

Towards Explorative IRBL: Combining Semantic Retrieval with LLM-Driven Iterative Code Exploration
Moumita Asad, Rafed Muhammad Yasir, and Sam Malek
(University of California at Irvine, USA)
Information Retrieval-based Bug Localization (IRBL) aims to identify buggy source files for a given bug report. Traditional and deep learning-based IRBL techniques often suffer from vocabulary mismatch and dependence on project-specific metadata. In contrast, recent Large Language Model (LLM)-based approaches struggle to provide appropriate context to the model: they either restrict analysis to a fixed set of candidate files, overwhelm the model with repository-wide information, or rely on explicit bug report cues to guide context collection. To address these issues, we propose GenLoc, a technique that combines semantic retrieval with LLM-driven code-exploration functions to iteratively analyze the code base and identify buggy files. We evaluate GenLoc on three complementary benchmarks, including large-scale and recent Java datasets as well as the Python based SWE-bench Lite dataset. Results demonstrate that GenLoc substantially outperforms traditional IRBL, deep learning-based approaches and recent LLM-based methods, while also localizing bugs that other techniques fail to detect.
Article Search Article: issta26main-p511-p (type: Full Paper) doi:10.1145/3832151

WASCII: Bridging WebAssembly Specifications and Implementations through LLM-Enhanced Validation
Yeqi Fu, Kaihang Ji, Yuanpeng Wang, Zong Cao, Jiahao Liu, Ding Li, Yao Guo, and Zhenkai Liang
(National University of Singapore, Singapore; Peking University, China; Imperial Global Singapore, Singapore; Nanyang Technological University, Singapore)
The rapid evolution of WebAssembly (Wasm) has led to significant implementation inconsistencies between its specification and the behavior of various Wasm runtimes, posing critical threats to application reliability and security. Verifying that a runtime's implementation adheres to the natural-language specification is a profound challenge. While Large Language Models offer a promising way to bridge the semantic gap between specification text and source code, their inherent fallibility makes them untrustworthy for direct verification. In this paper, we introduce WASCII, a novel framework for bridging specification and implementation with execution-based validation. Our approach first constructs a Check Tree from the natural-language specification, which captures the validation rules that runtimes must enforce. We then align runtime code to the Check Tree, and employ a Clean Room design with execution-based validation to ensure the correctness of the bridging. The validated test cases are then used for cross-runtime differential testing to identify behavioral inconsistencies. Evaluated on seven major Wasm runtimes, WASCII identified 248 differential behaviors, among which 35 are confirmed as previously unknown specification conformance issues, with 17 confirmed or fixed by developers. These results demonstrate that our approach is a highly effective strategy for discovering subtle yet critical bugs in complex systems.
Article Search Article: issta26main-p530-p (type: Full Paper) doi:10.1145/3832152

Automated Type-IV Clone Generation via LLMs and Deterministic Validation
Luciano Marchezan, Eugene Syriani, Kévin Delcourt, and Houari Sahraoui
(Université de Montréal, Canada)
Detecting Type-IV code clones, functionally equivalent fragments with different syntax, remains a major challenge for quality assurance. Existing datasets are limited in supporting semantic clone detection due to class imbalance, lack of verified functional equivalence, and data redundancy. We present an automated approach for generating Type-IV clones by leveraging large language models (LLMs) with deterministic testing and filtering. The approach normalizes input code, produces diverse clone candidates through customizable prompts, and ensures semantic equivalence via automated testing and syntactic diversity through CodeBLEU-based filtering. Representative unique clones are then selected by clustering. We evaluate the extent to which LLMs generate diverse Python Type-IV clones, how prompt and generation factors affect quality and efficiency, the retention of only Type-IV clones at the final dataset, and the usefulness of the resulting dataset for fine-tuning embedding models. Results show that the generated clones improve Type-IV clone detection across different programming languages.
Article Search Artifacts Available Article: issta26main-p553-p (type: Full Paper) doi:10.1145/3832153

Automated Type-IV Clone Generation via LLMs and Deterministic Validation (replication package) (doi:10.5281/zenodo.20858398): KaminoDatasetGenerator.zip - contains the replication package for the empirical evaluation. It includes the REQUIREMENTS, LICENSE, and STATUS files as requested for the AE track. Make sure to read the Readme.md inside the root folder for instructions. kamino_image.tar - is the docker image provided for reproducilibity ...

Signal Denoising Based Kill Matrix Refinement for Mutation-Based Fault Localization
Hengyuan Liu, Xia Song, Yong Liu, and Zheng Li
(Beijing University of Chemical Technology, China)
Software debugging is a critical and time-consuming aspect of software development, with fault localization being a fundamental step that significantly impacts debugging efficiency. Mutation-Based Fault Localization (MBFL) has gained prominence due to its robust theoretical foundations and fine-grained analysis capabilities. However, recent studies have identified a critical challenge: noise phenomena, specifically the false kill relationships between mutants and tests, which significantly degrade localization effectiveness. While several approaches have been proposed to rectify the final localization results, they do not directly address the underlying noise. In this paper, we propose a novel approach to refine the kill matrix, a core data structure capturing mutant-test relationships in MBFL, by treating it as a signal that contains both meaningful fault-related patterns and high-frequency noise. Inspired by signal processing theory, we introduce DKMR (Denoising-based Kill Matrix Refinement), which employs two key stages: (1) signal enhancement through hybrid matrix construction to improve the signal-to-noise ratio for better denoising, and (2) signal denoising via frequency domain filtering to suppress noise while preserving fault-related patterns. Building on this foundation, we develop MBFL-DKMR, a fault localization framework that utilizes the refined matrix with continuous values for suspiciousness calculation. Our evaluation on Defects4J v2.0.0 demonstrates that MBFL-DKMR effectively mitigates the noise and outperforms both state-of-the-art baselines (BLMu, Delta4Ms, and SMARTFL) and representative traditional baselines (MBFL_ME, MBFL_MU, and SBFL). Specifically, MBFL-DKMR localizes 141 faults at Top-1, compared to 113 for BLMu, 112 for Delta4Ms, and 101 for SMARTFL, while introducing negligible additional computational overhead (0.15 seconds, 0.0015% of total time).
Article Search Article: issta26main-p555-p (type: Full Paper) doi:10.1145/3832154

AutoHIL: LLM-Based ECU Functional Test Generation through Domain Knowledge Augmentation
Sichen Gong, Qicai Chen, Bihuan Chen, Wenzhuo Zhang, Yukun Gao, and Xin Peng
(Fudan University, China; ANHUI GuarDrive Safety Technology, China)
System functional testing is a critical step before the production release of automotive electronic control units (ECUs), especially for safety-critical functions. It is typically performed on hardware-in-the-loop (HIL) benches, and engineers translate functional requirements in natural language into executable test scripts using bench-specific APIs (or test harness primitives (THPs)). Existing research either focuses on generating abstract test cases that lack execution capability, or relies on bench-specific solutions that cannot be generalized. Consequently, industrial practice remains labor-intensive, and is hindered by three main challenges, i.e., the high learning cost of THPs, the incompleteness of functional requirements due to missing tacit domain knowledge (e.g., power management dependencies or system wake-up sequences), and the limited automation.
To address these challenges, we propose AUTOHIL, an LLM-based, domain-knowledge-augmented framework for requirement-driven ECU functional test generation on HIL benches. AUTOHIL automatically constructs a THP knowledge base to understand bench operations, extracts tacit domain knowledge from AUTOSAR ECU source code and configurations to augment requirements, and generates test scripts for the augmented requirements based on the THP knowledge base. We evaluate AUTOHIL on two industrial ECUs, i.e., an Airbag Control Unit (ACU) on an in-house bench and an Active Safety Domain Master (ASDM) on a dSPACE platform. AUTOHIL achieves test script executability of 90.33% on ACU and 81.31% on ASDM. We also manually evaluate the functional equivalence between generated scripts and manual scripts, and AUTOHIL reaches a functional correctness of 89.69% on ACU and 84.00% on ASDM. We discover 39 and 25 new functional defects in ACU and ASDM, which have been patched by technicians from the supplier.
Article Search Article: issta26main-p582-p (type: Full Paper) doi:10.1145/3832155

Atropos: Improving Cost-Benefit Trade-Off of LLM-Based Agents under Self-Consistency with Early Termination and Model Hotswap
Naryeong Kim and Shin Yoo
(KAIST, Republic of Korea)
Open-weight Small Language Models(SLMs) can provide faster local inference at lower financial cost, but may not achieve the same performance level as commercial Large Language Models (LLMs) that are orders of magnitudes larger. Consequently, many of the latest applications of LLMs, such as software engineering agents, tend to be evaluated on larger models only, leaving the issue of improving the cost-benefit trade-off of such applications neglected. This paper proposes Atropos, a predictive early-termination analysis and hotswap technique that aims to improve the cost-benefit trade-off for LLM-based agents that use self-consistency. The core component of Atropos is a predictive model based on structural properties of LLM inferences: after merging multiple agentic inference paths into a graph representation, Atropos uses Graph Convolutional Network (GCN) to predict whether an ongoing inference will eventually succeed or not. If an agentic task instance running on the source LLM is predicted to fail, Atropos subsequently performs hotswapping, i.e., migrating the on-going inference context onto the more capable target LLM: this is feasible because LLM contexts are stateless. An empirical evaluation of Atropos using three recent LLM-based agents shows that Atropos can predict early termination of eventually failing inferences with the accuracy of 0.85 at the midpoint of the inference. Hotswapping LLMs for such inferences can convert up to 27.57% of them to be successful. Consequently, Atropos achieves 74.35% of the performance of closed LLMs with as low as only 23.9% of the cost.
Article Search Article: issta26main-p589-p (type: Full Paper) doi:10.1145/3832156

On the Feasibility of Deduplicating Compiler Bugs with Bisection
Xintong Zhou, Zhenyang Xu, Yongqiang Tian, and Chengnian Sun
(University of Waterloo, Canada; Monash University, Australia)
Random testing has proven to be an effective technique for compiler validation. However, the debugging of bugs identified through random testing presents a significant challenge due to the frequent occurrence of duplicate test programs that expose identical compiler bugs. The process to identify duplicates is a practical research problem known as bug deduplication. Prior methodologies for compiler bug deduplication primarily rely on program analysis to extract bug-related features for duplicate identification, which can result in substantial computational overhead and limited generalizability.
This paper investigates the feasibility of employing bisection, a standard debugging procedure largely overlooked in prior research on compiler bug deduplication, for this purpose. Our study demonstrates that the utilization of bisection to locate failure-inducing commits provides a valuable criterion for deduplication, albeit one that requires supplementary techniques for more accurate identification. Building on these results, we introduce BugLens, a novel deduplication method that primarily uses bisection, enhanced by the identification of bug-triggering optimizations to minimize false negatives. Empirical evaluations conducted on five real-world datasets demonstrate that BugLens significantly outperforms the state-of-the-art analysis-based methodologies Tamer and D3 by saving an average of 33.56% and 10.68% human effort to identify the same number of distinct bugs. Given the inherent simplicity and generalizability of bisection, it presents a highly practical solution for compiler bug deduplication in real-world applications.
Article Search Article: issta26main-p601-p (type: Full Paper) doi:10.1145/3832157

Intrusion Models for Security Assessment: Methodology and Case Study on Xen
Charles Gonçalves and Marco Vieira
(University of Coimbra, Portugal; University of North Carolina at Charlotte, USA)
As cyberattacks become increasingly automated and amplified by emerging technologies, particularly Artificial Intelligence (AI), reliably assessing the security resilience of software systems becomes crucial. However, traditional methods centered on known vulnerabilities provide limited insight into attack impact. Intrusion Injection is an emerging approach that leverages Intrusion Models (IMs) to inject exploitable states representative of real intrusions, providing deeper insight into system resilience beyond known vulnerabilities. This paper formalizes Intrusion Models and proposes a structured methodology for their instantiation and injection into software systems. Grounded in fault-injection concepts, IMs define explicit abusive functionalities and the resulting erroneous states that, when injected, enable systematic analysis of software reliability under security threats. To demonstrate feasibility, we apply our approach to the Xen hypervisor, targeting memory-management and virtualization components using an injector prototype that allows security researchers and engineers to assess Xen-based systems for potential security-related failures. Our study indicates that intrusion injection driven by IMs can support repeatable, exploit-agnostic security assessments across platforms.
Article Search Article: issta26main-p643-p (type: Full Paper) doi:10.1145/3832158

Toward Secure Code Generation: Bridging Correctness and Security via Task-Adaptive Vulnerability Modeling and Execution-Based Benchmarking
Jiexin Wang, Liuwen Cao, Xitong Luo, Yang Cao, Zhenghao Li, Yunyi Xiao, Mengchen Zhao, Adam Jatowt, and Yi Cai
(South China University of Technology, China; University of Innsbruck, Austria)
Large language models (LLMs) are increasingly used for program synthesis, yet they often generate code that is functionally plausible but insecure. Progress in secure code generation has been hindered by benchmarks that are small, non-executable, leak mitigation details, or rely on noisy analyzers and subjective judgments, making it difficult to measure whether security improves without sacrificing correctness. We address these gaps with CodeSecEval, an execution-based benchmark for secure code generation, comprising 255 Python tasks spanning 77 CWE categories. Each task provides paired insecure and secure implementations together with executable functional and vulnerability-targeted security tests, enabling precise and reproducible evaluation of secure code generation and insecure-code repair. Building on CodeSecEval, we propose SecAwareCoder, an agent-based framework that shifts code generation toward secure-by-construction synthesis. SecAwareCoder performs task-adaptive threat modeling to identify security-sensitive regions and derive task-grounded vulnerability hypotheses, uses these hypotheses to guide both constraint-aware code generation and security-aware test synthesis, and leverages execution feedback for targeted refinement. Experiments across multiple LLM backbones show that SecAwareCoder consistently improves Pass@1 and security robustness over prompting and analyzer-driven baselines, narrowing the security--correctness gap in LLM code generation.
Article Search Article: issta26main-p648-p (type: Full Paper) doi:10.1145/3832159

Fuzzing FPGA Synthesis and Simulation Tools via LLM-Generated Syntax-Valid HDL Codes
He Jiang, Wen Zhao, Shikai Guo, Zhihao Xu, Xiaochen Li, and Rubing Huang
(Dalian University of Technology, China; Dalian Maritime University, China; Southeast University, China; Macau University of Science and Technology, China)
Field-Programmable Gate Array (FPGA) synthesis and simulation tools, such as Vivado, Quartus, Yosys, and Icarus Verilog, are key components of Electronic Design Automation (EDA) toolchains, translating high-level Hardware Description Language (HDL) designs into low-level gate netlists. However, defects in these compilers can propagate into the synthesized netlists, leading to crashes and functionally incorrect or even insecure hardware implementations and posing significant security risks. Existing fuzz testing approaches face several challenges, including limited diversity in primitive-cell types and a lack of feedback-guided exploration. These issues restrict their ability to thoroughly exercise the compilers and expose deep-seated defects.
To address these challenges, we propose PolyHDL, which leverages the prompting Large Language Models (LLMs) for generating valid HDL designs to detect compiler defects in FPGA synthesis and simulation tools. By leveraging prompt learning and integrating feedback-driven guidance from primitive-cell diversity, PolyHDL generates semantically valid HDL designs with diverse primitive-cell combinations, thereby addressing the aforementioned challenges. Furthermore, through equivalence check, PolyHDL effectively reveals potential compiler defects in FPGA synthesis and simulation tools. Experimental results demonstrate that PolyHDL successfully identified and reported 18 valid defects in widely used toolchains, including Vivado, Yosys, Icarus Verilog, and Quartus within one month, 17 of which were confirmed by the official technical support, and achieved a 13.1%–13.4% improvement in code coverage over the SOTA approaches.
Article Search Article: issta26main-p653-p (type: Full Paper) doi:10.1145/3832160

AutoCodeSherpa: Symbolic Explanations in AI Coding Agents
Sungmin Kang, Haifeng Ruan, and Abhik Roychoudhury
(National University of Singapore, Singapore)
Large language model (LLM) agents integrate external tools with one or more LLMs to accomplish specific tasks. Agents have rapidly been adopted by developers, and they are starting to be deployed in industrial workflows, such as their use to fix static analysis issues from the widely used SonarQube static analyzer. However, the growing importance of agents means their actions carry greater impact and potential risk. Thus, to use them at scale, an additional layer of trust and evidence is necessary. This work presents AutoCodeSherpa, a technique that provides explanations of software issues in the form of symbolic formulae. Inspired by the reachability, infection, and propagation model of software faults, the explanations are composed of input, infection, and output conditions, collectively providing a specification of the issue. In practice, the symbolic explanation is implemented as a combination of a property-based test (PBT) and program-internal symbolic expressions. Critically, this means our symbolic explanations are executable and can be automatically evaluated, unlike natural language explanations. Experiments show the generated conditions are highly accurate. For example, input conditions from AutoCodeSherpa had an accuracy of 85.7%. This high accuracy makes symbolic explanations particularly useful in two scenarios. First, the explanations can be used in automated issue resolution environments to decide whether to accept or reject patches from issue resolution agents; AutoCodeSherpa could reject 2x as many incorrect patches as baselines did. Second, as agentic AI approaches continue to develop, program analysis driven explanations like ours can be provided to other LLM-based repair techniques which do not employ analysis to improve their output. In our experiments, our symbolic explanations could improve the plausible patch generation rate of the Agentless technique by 60%.
Article Search Artifacts Available Article: issta26main-p671-p (type: Full Paper) doi:10.1145/3832161

Artifact (doi:10.5281/zenodo.21900208): The artifact contains the source code, experimental scripts, and evaluation results of the paper.

From Greedy Steps to Global Optimization: Learning Sequential Test Suite Generation
Guoqing Wang, Chengran Yang, Xiaoxuan Zhou, Zeyu Sun, Bo Wang, David Lo, and Dan Hao
(Peking University, China; Singapore Management University, Singapore; Northeastern University, China; Institute of Software at Chinese Academy of Sciences, China; Beijing Jiaotong University, China)
With the rapid evolution of Large Language Models (LLMs), automated software testing is witnessing a paradigm shift. While proprietary models like GPT-4o demonstrate impressive capabilities, their high deployment costs and data privacy concerns make open-source LLMs the practical imperative for many academic and industrial scenarios. In the field of automated test generation, it has evolved to iterative workflows to construct test suites based on LLMs. When utilizing open-source LLMs, we empirically observe they lack a suite-level perspective, suffering from structural myopia—failing to generate new tests with large marginal gain based on the current covered status. In this paper, from the perspective of sequences, we formalize test suite generation as a Markov Decision Process (MDP) and demonstrate that its objective exhibits monotone submodularity, which enables an effective relaxation of this NP-hard global optimization into a tractable step-wise greedy procedure. Guided by this insight, we propose TestDecision, which transforms LLMs into neural greedy experts. TestDecision consists of two synergistic components: (1) an inference framework which implements test suite construction following a step-wise greedy strategy; and (2) a training pipeline of reinforcement learning which equips the base LLM with sequential test generation ability to maximize marginal gain. Comprehensive evaluations on the ULT benchmark demonstrate that TestDecision significantly outperforms existing advanced methods. It brings an improvement between 38.15-52.37% in branch coverage and 298.22-558.88% in execution pass rate over all base models, achieving a comparable performance on 7B backbone with a much larger proprietary LLM GPT-5.2. Furthermore, TestDecision can find 58.43-95.45% more bugs than vanilla base LLMs and exhibit superior generalization on LiveCodeBench, proving its capability to construct high-quality test suites.
Article Search Article: issta26main-p699-p (type: Full Paper) doi:10.1145/3832162

SymWeb: Feedback-Driven Context Exploration and Context-Aware Symbolic Execution for Browser-Embedded WebAssembly Vulnerability Detection
Yuanpeng Wang, Yeqi Fu, Zhineng Zhong, Zhenkai Liang, Ding Li, Yao Guo, and Xiangqun Chen
(Peking University, China; National University of Singapore, Singapore)
Browser-deployed WebAssembly (Wasm) modules often inherit memory-safety bugs from C and C++-style code, yet exploiting, and even reaching, these bugs in the Web threat model is fundamentally context-dependent. JavaScript (JS) controls the exported-call schedule and constructs the Wasm entry state, including arguments, globals, and linear-memory layouts, from attacker-influenced web inputs. This makes both Wasm-only analysis, which assumes static initial states, and prior browser-based testing such as Wemby ineffective. Wemby generates a fixed, Wasm-agnostic context pool and then only mutates Wasm parameters, which limits its ability to systematically reach deeper, Wasm-relevant contexts and gated behaviors.
We present SymWeb, a feedback-driven closed-loop system that links external inputs to browser-reachable JS-induced Wasm contexts and then to context-aware Wasm symbolic execution. SymWeb couples an Feedback-driven Context Generator with an Context-Aware Wasm Symbolic Executor. The Feedback-driven Context Generator performs binary rewriting for ASan-like checks and observability, collects contexts in the browser, and uses Influence-guided Mutation to steer web inputs. The symbolic executor clusters and symbolizes contexts, performs coverage-guided symbolic execution under reachable entry states, and returns actionable constraints to steer the next online round.
We evaluate SymWeb on 30 real-world Wasm-enabled websites. Under our Web threat model, SymWeb verifies 17 exploitable vulnerabilities and achieves 72.8% average Wasm basic-block coverage. Compared to the browser-based baseline Wemby, SymWeb finds 8 more verified vulnerabilities and improves coverage by 19.9 percentage points. Compared to the Wasm-only baseline WASEM, SymWeb finds 14 more verified vulnerabilities and improves coverage by 40.4 percentage points. Overall, these results show that closing the loop between browser-reachable context generation and context-aware Wasm analysis substantially improves both vulnerability-finding effectiveness and exploration depth in real Web environments.
Article Search Artifacts Available Article: issta26main-p700-p (type: Full Paper) doi:10.1145/3832163

Artifact for Ariticle 'SymWeb: Feedback-Driven Context Exploration and Context-Aware Symbolic Execution for Browser-Embedded WebAssembly Vulnerability Detection' (doi:10.5281/zenodo.21309545): SymWeb is the research artifact accompanying the paper “SymWeb: Feedback-Driven Context Exploration and Context-Aware Symbolic Execution for Browser-Embedded WebAssembly Vulnerability Detection.” It implements a feedback-driven closed-loop system for detecting memory-safety vulnerabilities in browser-embedded ...

The Unseen Delta: Characterizing the Compiler Optimization Landscape via Top-Down Differential Analysis
Zhibo Liu, Huaijin Wang, and Shuai Wang
(Nanjing University, China; Shandong University, China; Hong Kong University of Science and Technology, China)
Compiler optimizations are essential for achieving high performance in modern software. However, recent studies highlight the persistence of performance bugs, i.e., subtle defects where the compiler generates functionally correct but computationally inefficient code, leading to significant performance degradation. Existing detection and testing methods typically employ a bottom-up approach, focusing on specific low-level code properties and remaining confined to known optimization rules. Consequently, they struggle to quantify the holistic impact of identified issues and often overlook critical microarchitectural inefficiencies.
We observe a key indicator of untapped potential: different compilers often produce binaries with significant performance differences for identical source code. However, the root causes of these discrepancies remain largely unexplored and difficult to pinpoint using current techniques. To bridge this gap, we introduce a top-down differential analysis methodology. This approach calibrates compiler optimization differences with fine-grained, hierarchical microarchitectural metrics, offering a comprehensive view of runtime behavior. Using a sampling-based approach, this method efficiently pinpoints the critical code snippets responsible for performance differences, enabling targeted root cause analysis.
Our empirical evaluation uncovers substantial and often surprising performance differences between binaries generated by GCC and Clang. A categorization of root causes reveals systemic challenges in compiler optimizations. To quantitatively validate our findings and demonstrate practical impact, we developed a binary patching framework that fixes identified performance issues by transplanting superior code sequences from competing compilers. This work provides a novel lens for understanding and analyzing optimization defects.
Article Search Article: issta26main-p711-p (type: Full Paper) doi:10.1145/3832164

AgentBreaker: Evaluating Context-Aware Indirect Prompt Injection Risks in Modern Web Agents
Yongbi Son, Changoo Lee, Dongwon Shin, Byoungyoung Lee, Sanghyun Hong, and Sooel Son
(KAIST, Republic of Korea; Seoul National University, Republic of Korea; Oregon State University, USA)
Recent advances in large language models (LLMs) have enabled autonomous web agents to perform complex user tasks by leveraging their adaptive decision-making capabilities. Despite their growing use in crawling the Web, their security implications under indirect prompt injection (IPI) attacks remain largely understudied. Prior studies have compiled static benchmarks or proposed dynamic frameworks that generate adversarial phrases aimed at deceiving a single LLM within a target agent. However, by ignoring the agent’s operating context, these approaches yield suboptimal IPI attacks against modern web agents leveraging multiple, specialized LLMs. In this paper, we study the vulnerability in web agents to malicious phrases embedded as HTML elements. To assess the security risks posed by this vulnerability, we present AgentBreaker, an IPI attack framework that autonomously composes adversarial phrases tailored to page-specific context. When processed by web agents, these DOM-embedded phrases induce adversarial behaviors, such as clicking attacker-designated HTML elements, posting attacker-provided text, and disclosing internal agent secrets. In our evaluation against five state-of-the-art web agents, AgentBreaker achieves an attack success rate of 71.7%–100% across 60 webpages sampled from Online-Mind2Web. We then propose practical defenses that not only mitigate observed threats but also address potential adaptive attacks. Our defenses reduce the attack success rate down to 1.7%. By conducting context-aware injection, AgentBreaker outperforms existing IPI frameworks, thereby accurately evaluating web agents’ susceptibility to IPI and providing stepping stones for countering this emerging threat.
Article Search Artifacts Available Article: issta26main-p713-p (type: Full Paper) doi:10.1145/3832165

AgentBreaker: Evaluating Context-Aware Indirect Prompt Injection Risks in Modern Web Agents (Artifact) (doi:10.6084/m9.figshare.32835578.v2): LLM-powered web agents are already deployed at consumer scale. Despite their increasing use for web automation, their security implications under Indirect Prompt Injection (IPI) attacks remain largely understudied, while existing attack methods achieve relatively low attack success rates (ASRs). AgentBreaker is an ...

Automated Modernization of Machine Learning Engineering Notebooks for Reproducibility
Bihui Jin, Kaiyuan Wang, and Pengyu Nie
(University of Waterloo, Canada; Google, USA)
Interactive computational notebooks (e.g., Jupyter notebooks) are widely used in machine learning engineering (MLE) to program and share end-to-end pipelines, from data preparation to model training and evaluation. However, environmental erosion—the rapid evolution of hardware and software ecosystems for machine learning—has rendered many published MLE notebooks non-reproducible in contemporary environments, hindering code reuse and scientific progress. To quantify this gap, we study 12,106 notebooks selected from 75 popular Kaggle competitions: only 26% remain reproducible today. Crucially, we find that environment backporting, i.e., downgrading dependencies to match the submission time, does not improve reproducibility (decreased to 12%) but rather introduces additional failure modes.
To address environmental erosion, we design and implement MLEModernizer, an LLM-driven agentic framework that treats the contemporary environment as a fixed constraint and modernizes notebook code to restore reproducibility. MLEModernizer iteratively executes notebooks, collects execution feedback, and applies three types of targeted fixes: error-repair, runtime-reduction, and score-calibration. Evaluated on 8,210 notebooks that are non-reproducible under the baseline environment, MLEModernizer makes 3,292 (40.1%, GPT-5.2) and 3,683 (44.9%, GPT-OSS-120b) notebooks reproducible. MLEModernizer presents a best-effort automated recovery and modernization technique that can improve reproducibility for a subset of notebooks. Practitioners can leverage MLEModernizer to validate, reuse, and maintain MLE artifacts as the hardware and software ecosystems continue to evolve.
Article Search Article: issta26main-p725-p (type: Full Paper) doi:10.1145/3832166

Seeing Is Coding: On the Effectiveness of Vision Language Models in Code Understanding
Yuling Shi, Chaoxiang Xie, Zhensu Sun, Yeheng Chen, Chenxu Zhang, Longfei Yun, Chengcheng Wan, Hongyu Zhang, David Lo, and Xiaodong Gu
(Shanghai Jiao Tong University, China; Hohai University, China; Singapore Management University, Singapore; Imperial College London, UK; University of California at San Diego, USA; East China Normal University, China; Shanghai Innovation Institute, China; Chongqing University, China)
Large Language Models (LLMs) have achieved remarkable success in source code understanding, yet as software systems grow in scale, computational efficiency has become a critical bottleneck. Currently, these models rely on a text-based paradigm that treats source code as a linear sequence of tokens, which leads to a linear increase in context length and associated computational costs. The rapid advancement of Multimodal LLMs (MLLMs) introduces an opportunity to optimize efficiency by representing source code as rendered images. Unlike text, which is difficult to compress without losing semantic meaning, the image modality is inherently suitable for compression. By adjusting resolution, images can be scaled to a fraction of their original token cost while remaining recognizable to vision-capable models. To explore the feasibility of this approach, we conduct the first systematic study on the effectiveness of MLLMs for code understanding. Our experiments reveal that: (1) MLLMs can effectively understand code with substantial token reduction, achieving up to 8× compression; (2) MLLMs can effectively leverage visual cues such as syntax highlighting, improving code completion performance under 4× compression; and (3) Code-understanding tasks like clone detection exhibit exceptional resilience to visual compression, with some compression ratios even slightly outperforming raw text inputs. Our findings highlight both the potential and current limitations of MLLMs in code understanding, which points out a shift toward image-modality code representation as a pathway to more efficient inference.
Article Search Article: issta26main-p726-p (type: Full Paper) doi:10.1145/3832167

ConFL: Explainable Concurrent Fault Localization via Hierarchy-Guided LLM Reasoning
Shuai Shao, Dingbang Wang, Yiming Zeng, and Tingting Yu
(University of Connecticut, USA)
Localizing concurrent bugs from bug reports alone is challenging due to incomplete information, misleading program-entity mentions, and complex cross-thread interactions, causing existing LLM-based approaches to suffer from unstable reasoning and limited explainability. We propose ConFL, an explainable concurrent fault localization framework that augments LLM reasoning with structured concurrency knowledge. ConFL constructs a Concurrent Knowledge Base (CKB) from source code and performs LLM-guided hierarchical retrieval to progressively narrow the search space from components to interaction-level concurrency contexts. An interaction-level DSL explicitly encodes cross-thread interactions over shared resources, enabling focused reasoning without traversing deep call chains. Experiments on real-world concurrent bugs from eight large-scale Java projects show that ConFL significantly outperforms state-of-the-art IR-based and LLM-based baselines, achieving an MRR of 0.503 and a MAP of 0.486, while remaining robust to noisy bug reports, unseen bugs, and different LLM backbones.
Article Search Article: issta26main-p727-p (type: Full Paper) doi:10.1145/3832168

Belobog: Move Language Fuzzing Framework for Real-World Smart Contracts
Ziqiao Kong, Wanxu Xia, Zhengwei Li, Yi Lu, Pan Li, Liqun Yang, Yang Liu, Xiapu Luo, and Shaohua Li
(Nanyang Technological University, Singapore; Beihang University, China; Bitslab, Singapore; Movebit, Singapore; Hong Kong Polytechnic University, Hong Kong; Chinese University of Hong Kong, Hong Kong)
Move is a resource-oriented programming language designed for secure and verifiable smart contract development and has been widely used in managing billions of digital assets in blockchains, such as Sui and Aptos.Move features a strong static type system and explicit resource semantics to enforce safety properties such as the prevention of data races, invalid asset transfers, and entry vulnerabilities. However, smart contracts written in Move may still contain certain vulnerabilities that are beyond the reach of its type system. It is thus essential to validate Move smart contracts. Unfortunately, due to its strong type system, existing smart contract fuzzers are ineffective in producing syntactically or semantically valid transactions to test Move smart contracts.
This paper introduces the first fuzzing framework, Belobog, for Move smart contracts. Belobog is type-aware and ensures that all generated and mutated transactions are well-typed. More specifically, for a target Move smart contract, Belobog first constructs a dependency graph based on Move's type system, and then generates or mutates a transaction based on the graph trace derived from the dependency graph. In order to overcome the complex checks in Move smart contracts, we further design and implement a concolic executor in Belobog.
We evaluated Belobog on 109 real-world Move smart contract projects. The experimental results show that Belobog is able to detect 100% critical and 79% major vulnerabilities manually audited by human experts. We further selected two recent notorious incidents in the Move ecosystem, i.e., Cetus and Nemo. Belobog successfully reproduced full exploits for both of them, without any prior knowledge. Moreover, we applied Belobog on three ongoing auditing projects and found 2 critical, 2 major, and 3 medium new vulnerabilities, all acknowledged by the project developers.
Article Search Artifacts Available Article: issta26main-p733-p (type: Full Paper) doi:10.1145/3832169

Artifact of "BELOBOG: Move Language Fuzzing Framework for Real-World Smart Contracts" (doi:10.5281/zenodo.21062917): This artifact accompanies the paper "Belobog: Move Language Fuzzing Framework for Real-World Smart Contracts". The artifact contains two parts: (1) a source-code archive for Belobog, including the Rust and Move source code and README; and (2) a prebuilt Docker image for reproducing the public experiments on Sui and ...

Insecure Coding Preferences in Long-Term Memory: Security Risks for LLM-Based Code Generation
Yuchen Chen, Wei Cheng, Yuan Xiao, Zhou Yang, Weifeng Sun, Chunrong Fang, Xiang Chen, Baowen Xu, David Lo, and Zhenyu Chen
(Nanjing University, China; Nanjing University of Aeronautics and Astronautics, China; University of Alberta, Canada; Alberta Machine Intelligence Institute, Canada; Singapore Management University, Singapore; Nantong University, China)
LLM-based systems increasingly incorporate long-term memory to improve cross-session continuity. However, once insecure coding preferences are stored, they may silently influence security-critical decisions in subsequent generations. In this study, we conduct the first systematic empirical study on the impact of insecure coding preferences stored in long-term memory on the security of LLM-based code generation. We evaluate four LLMs (ChatGPT, Gemini, Qwen, and Grok) across five programming languages (Python, C, C++, Go, and JavaScript). Our results show that insecure memories significantly increase the risk of generating vulnerable code by 2.7-50.3 percentage points (pp). Moreover, they create a 5.4-14.0 percentage-point risk-warning gap, where warning-rate increases lag behind vulnerability-rate increases. Further analysis reveals that insecure memories are difficult to overwrite through normal interactions and can broadly influence model outputs even when prompts are phrased differently. Finally, we evaluate three mitigation strategies: security-requirement appending and memory storage reduce vulnerability rates by 19.7-33.6 pp but may degrade functional correctness by up to 15.9 pp; memory-level safety filtering achieves a 100% detection rate on our evaluated risky memory entries and restores generation behavior to the without-memory baseline. Based on these findings, we provide actionable suggestions to improve the security of long-term memory in LLM-based code generation.
Article Search Article: issta26main-p747-p (type: Full Paper) doi:10.1145/3832170

TraceDev: A Traceability-Driven Multi-agent Framework for Requirement-to-Code Development
Mingyu Chen, Yakun Zhang, Zihao Xie, Yixing Luo, Jinrui Xu, Cuiyun Gao, Kaiqi Zhao, and Yunming Ye
(Harbin Institute of Technology, Shenzhen, China; Beijing Institute of Control Engineering, China)
In modern software development, the rapid advancement of Large Language Models (LLMs) has made the end-to-end transformation of Natural Language Requirements (NLRs) into executable repository-level code increasingly feasible. However, existing approaches typically rely on simplified instructions (e.g., single-sentence descriptions), failing to reflect complex software development scenarios. Moreover, they lack explicit requirement traceability mechanisms, making it difficult to precisely align and validate generated code against original requirements. To address these limitations, we propose TraceDev, a multi-agent framework for automated software development grounded in use cases that contain multiple functional points and complex semantics. TraceDev employs five role-specific agents, including a Requirement Refiner, Designer, Developer, Tester, and Validator. Notably, the Validator Agent constructs and maintains a heterogeneous traceability graph that links requirements, design models, and code artifacts for interacting with the preceding four agents. The traceability graph maintains consistency across various artifacts and serves as a structured context for efficient memory management, supporting reliable repository-level code generation. We evaluate TraceDev on two widely used datasets (including 125 use cases) compared with two state-of-the-art approaches. On the ETOUR dataset, TraceDev achieves a success rate of 53.63%, outperforming baseline approaches by up to 186.63%. A similar trend is observed on the SMOS dataset, where TraceDev attains a success rate of 56.82%, exceeding baseline approaches by up to 340.80%. These results demonstrate the effectiveness of TraceDev in repository-level code generation from requirements.
Article Search Article: issta26main-p753-p (type: Full Paper) doi:10.1145/3832171

Efficient Grammar-Constrained Decoding via Parser Stack Classification
Yongmin Li, Yihong Dong, Jia Li, and Ge Li
(Peking University, China; Shanghai Jiao Tong University, China; Tsinghua University, China)
LLMs are widely used to generate structured output like source code or JSON. Grammar-constrained decoding (GCD) can guarantee the syntactic validity of the generated output, by masking out tokens that violate rules specified by a context-free grammar. However, the online computational overhead of existing GCD methods, with latency typically scaling linearly with vocabulary size, limits the throughput of LLMs, especially for models with large vocabularies. To address this issue, we propose PSC, a novel grammar-constrained decoding method. By combining acceptance conditions of all vocabulary tokens into a single classifier of the parser stack during preprocessing, PSC can compute the complete vocabulary mask by checking the parser stack exactly once per decoding step, with time complexity independent of the vocabulary size. Experiments show that PSC computes masks up to 700× faster than baselines on complex programming language grammars, and up to 30× faster for schema-conformant JSON; end-to-end LLM throughput with PSC approaches that of unconstrained decoding. We analyze the preprocessing overhead for preprocessing providers and decoding users, and provide a break-even point analysis to help users decide whether to do preprocessing by themselves.
Article Search Article: issta26main-p756-p (type: Full Paper) doi:10.1145/3832172

MoT: Modularization-of-Thought Prompting for Effective Code Generation
Ruwei Pan and Hongyu Zhang
(Chongqing University, China)
Large Language Models are transforming software development by automatically generating code. Current prompting techniques such as Chain-of-Thought (CoT) suggest tasks step by step and the reasoning process follows a linear structure, which hampers the understanding of complex programming problems, particularly those requiring hierarchical solutions. Inspired by the principle of modularization in software development, in this work, we propose a novel prompting technique called MoT (Modularization of Thought), to enhance the code generation performance of LLMs. First, MoT exploits modularization principles to decompose complex programming problems into smaller, independent reasoning steps, enabling a more structured and interpretable problem-solving process. This hierarchical structure improves LLMs' ability to comprehend complex programming problems. Then, it structures the reasoning process using an MLR Graph (Multi-Level Reasoning Graph), which hierarchically organizes reasoning steps. This approach enhances modular understanding and ensures better alignment between reasoning steps and the generated code, significantly improving code generation performance. Our experiments on two advanced LLMs (GPT-4o-mini and DeepSeek-R1), comparing MoT to six baseline prompting techniques across eight benchmarks, demonstrate that MoT significantly outperforms existing baselines (e.g., CoT and SCoT), achieving Pass@1 scores ranging from 58.1% to 95.1%.
Article Search Article: issta26main-p757-p (type: Full Paper) doi:10.1145/3832173

AgentInspect: Diagnosing Behavioral Failures in Artificial Intelligence Agents
Ruchira Manke, Mohammad Wardat, Foutse Khomh, and Hridesh Rajan
(Tulane University, USA; Oakland University, USA; Polytechnique Montreal, Canada)
Effectively testing Artificial Intelligence (AI) agents remains a fundamental challenge due to their stochastic reasoning, vast and diverse input space, reliance on external tools, and operation in dynamic execution environments; factors that demand new testing methodologies explicitly tailored to the complex and interactive nature of agent-based systems. This work presents a novel methodology for testing AI agents, with a particular focus on assessing their behavioral robustness under varied operational conditions. Our approach relies on following key technical innovations: (1) a coverage-guided test input generation strategy based on agent- specific coverage objectives, (2) a capture-and-simulate mechanism that systematically emulates abnormal tool behaviors to mimic real-world execution failures, and (3) a deterministic behavioral failure detection approach that enables consistent identification of failures across different test inputs. We developed AgentInspect, a framework that automatically detects six types of behavioral failures in LangChain-based AI agents by analyzing their execution trajectories across three evaluation settings: a baseline setting using real tool responses, a simulated setting incorporating synthetic tool responses, and a hybrid setting that combines the real and simulated tool responses. To evaluate our approach, we curated a benchmark of 35 AI agents obtained from GitHub. Our results show that AgentInspect consistently identifies different behavioral failures with high precision and recall across all three execution settings. In particular, the simulated and hybrid settings expose failure modes that do not emerge during baseline execution with real tool responses, thereby enabling a more comprehensive assessment of agent robustness. Our findings highlight AgentInspect’s effectiveness in revealing critical failures and its practical utility for systematic robustness evaluation of AI agents.
Article Search Article: issta26main-p761-p (type: Full Paper) doi:10.1145/3832174

Hidden Licensing Risks in the PTMware Ecosystem
Bo Wang, Yueyang Chen, Jieke Shi, Minghui Li, Yunbo Lyu, Yinan Wu, Youfang Lin, and Zhou Yang
(Beijing Jiaotong University, China; Singapore Management University, Singapore; North Carolina State University, USA; University of Alberta, Canada; Alberta Machine Intelligence Institute, Canada)
Pre-trained Models (PTMs) have been increasingly integrated into software systems, giving rise to a new class of software referred to as PTMware. In addition to traditional software components composed solely of source code, PTMware also embeds or interacts with PTMs that depend on other models and datasets, forming complex supply chains involving open-source software (OSS) libraries, PTMs, and datasets. However, the licensing issues arising from these intertwined dependencies remain largely unexplored. Leveraging GitHub and Hugging Face, two premier hubs for code and models, we curate a large-scale dataset capturing the supply chains of PTMware. Our dataset comprises 12,180 OSS repositories from GitHub, 3,988 PTMs, and 708 datasets from Hugging Face. We analyze license distributions in the PTMware ecosystem and find that licensing practices differ markedly from those in traditional OSS communities. We further examine license-related issues and identify license selection and maintenance as the primary pain points, with 84% of cases involving discussions about adding appropriate licenses or resolving conflicts in existing ones. We then study license incompatibility in PTMware and evaluate the state-of-the-art approaches, finding that they perform poorly in this setting and achieve only 58% and 76% F1 scores, respectively. These results motivate us to propose LiAgent, which explores the potential of LLM-based agents for ecosystem-level license compatibility analysis, achieves an F1 score of 87%, and improves performance by 14 percentage points over prior approaches. We submit 60 license incompatibility issues detected by LiAgent, of which developers have confirmed 11. Two PTMs with license conflicts have more than 107 million and 5 million downloads on Hugging Face, respectively, suggesting that the issues may affect many downstream applications. We conclude by discussing implications and providing recommendations to support the healthy growth of the PTMware ecosystem.
Article Search Artifacts Available Article: issta26main-p764-p (type: Full Paper) doi:10.1145/3832175

Artifact for "Hidden Licensing Risks in the PTMware Ecosystem" (doi:10.5281/zenodo.21061130): The data and code for ISSTA-26 paper "Hidden Licensing Risks in the PTMware Ecosystem".

Finding and Understanding Missed Optimizations in WebAssembly Optimizer (Experience Paper)
Ruiyang Xu, Zetao Fan, Shan Huang, and Ting Su
(East China Normal University, China)
As WebAssembly (Wasm) expands from web applications to high-performance domains, the standard optimizer, wasm-opt, is critical but frequently suffers from missed optimizations (MOs). This paper presents an experience report on detecting MOs in wasm-opt and understanding their root cause through the lens of Wasm’s tree-structured intermediate representation (tree IR). To this end, we adapt an established marker-based technique from C compilers, overcoming the constraints of Wasm’s structured control flow via a novel structure-aware instrumentation strategy. Complementing this, we design a cross-optimization differential testing strategy leveraging the monotonicity of optimization levels as an oracle. Together, these strategies enable the systematic identification of fine-grained MOs that are overlooked by existing cross-architecture methods. Our evaluation uncovered 24 distinct MOs (20 fixed, 100% confirmation rate, 0% false positive rate), demonstrating the high actionability and effectiveness of our approach. The performance impact, especially in code size, yields an average 1.30% improvement and no regressions on the Emscripten benchmark suite, further proving their practical value. Beyond detection, our analysis distills three practical lessons for designing MO testing techniques and understanding the optimization trade-offs imposed by wasm-opt’s tree IR.
Article Search Article: issta26main-p770-p (type: Full Paper) doi:10.1145/3832176

Are We Stuck? Modeling and Detecting Deadlocks in Multi-autonomous Vehicle Systems
Mingfei Cheng, Xiaofei Xie, Lili Quan, and Yuan Zhou
(Singapore Management University, Singapore; Zhejiang Sci-Tech University, China)
Autonomous driving system (ADS) testing is essential to ensure the safety and reliability of autonomous vehicles (AVs) prior to deployment. As ADSs are increasingly deployed in multi-AV traffic environments, it becomes crucial to assess their cooperative performance, particularly with respect to deadlock, a fundamental liveness issue in concurrent systems that can lead to traffic congestion and prolonged stalling. However, the analysis and testing of ADSs’ cooperative capabilities with respect to deadlock remain largely underexplored. In this work, we present the first systematic study of deadlock in multi-AV systems. We formalize deadlock in autonomous driving using a time-indexed wait-for relation grounded in vehicles’ planned trajectories and road-region occupancy. Building on this formalization, we propose WaitWatch, a wait-for-oriented testing framework that steers scenario generation via spatio-temporal intersection alignment of executed trajectories to induce circular wait patterns. WaitWatch integrates a Deadlock Judge, Intersection Alignment Feedback, and Intersection Oriented Mutation to efficiently uncover latent deadlock scenarios. We conduct an extensive evaluation on three representative ADSs. Experimental results show that, on average, WaitWatch generates 2.28× as many deadlock scenarios (DLSs) as the best-performing baseline. By shifting the focus from single- AV evaluation to multi-AV cooperation, our approach identifies a range of previously unknown deadlock behaviors, revealing significant limitations in the cooperative and liveness capabilities of current ADSs. Our findings highlight a fundamental safety–liveness trade-off in deadlock resolution and demonstrate the need for systematic deadlock-aware testing in the development and validation of autonomous driving systems.
Article Search Article: issta26main-p772-p (type: Full Paper) doi:10.1145/3832177

StateTree: A Tree-Based Modeling Approach for Fault Detection in Recurrent Neural Networks
Xinyu Gao, Shuoxiao Zhang, Minghui Wei, Xiao Zhang, An Guo, and Enyi Tang
(Nanjing University, China; Hong Kong Polytechnic University, China)
Recurrent Neural Networks (RNNs) have become a core component of modern intelligent software due to their strong ability to model temporal dependencies. As RNNs are increasingly deployed in safety-critical domains, ensuring their reliability is crucial. However, most existing testing techniques are designed for feedforward networks and struggle with RNNs. The stateful nature, recurrent feedback, and long-term dependencies of RNNs make it difficult for existing testing methods to capture decision logic and temporal behaviors, which in turn makes detecting fault-inducing behaviors that emerge through temporal decision evolution challenging.
To address these challenges, we propose StateTree, a tree-based abstract modeling approach for systematic testing of RNN-based systems. StateTree constructs an Abstract State Tree (AST) that captures major RNN decision behaviors, where each root-to-leaf path represents an abstract decision for intuitive interpretation and structured exploration. Using the AST, StateTree guides the testing process toward both major and previously unseen paths to reveal erroneous behaviors. Experiments show that StateTree accurately abstracts RNN decisions, detects hundreds of faults, and retraining with its identified test cases improves robustness beyond existing RNN coverage methods, demonstrating its effectiveness in both fault detection and model performance enhancement.
Article Search Article: issta26main-p774-p (type: Full Paper) doi:10.1145/3832178

Poirot: Automatic Root Cause Analysis of Safety Violations in ADS Simulation Testing via Hypothetical Reasoning
You Lu, Dingji Wang, Kun Zhang, Bihuan Chen, Jiyan Zhang, and Xin Peng
(Fudan University, China)
With the rapid development of autonomous driving systems (ADSs), it has become critical to ensure their operational safety, leading to the widespread adoption of simulation testing. While existing scenario-based simulation testing approaches have demonstrated effectiveness in detecting safety violations, they often fall short in providing insight into the underlying causes of these violations, which is an essential capability for improving the safety and reliability of ADSs. To address this limitation, we propose a two-phase novel framework, Poirot, for root cause analysis in simulation testing via hypothetical reasoning. Given a reproducible violation scenario, in the module-level analysis phase, Poirot replays the violation scenario and identifies the faulty module by iteratively replacing an actual module with an idealized module and checking whether the violation persists. In the component-level analysis phase, depending on the identified faulty module, Poirot further applies either hypothetical reasoning with a suspicion-guided search strategy or causal analysis to narrow the fault space and pinpoint the faulty component. We evaluate Poirot with two ADSs, e.g., Apollo and Autoware, on a comprehensive benchmark that includes a total of 80 real and injected faults along with their triggering scenarios. Compared with the state-of-the-art root cause analysis approaches, e.g., ACAV and Rocas, Poirot improves the module-level accuracy by 187.29% on average, and identifies the faulty components at a finer granularity, achieving component-level accuracy of 90.62%. Our ablation study shows that our suspicion-guided search strategy in Poirot efficiently reduces the exploration of the fault space by 58.77%, leading to a 65.41% reduction in the time for fault localization. Finally, applied to two scenario-based simulation testing methods, i.e., AvFuzzer and MoDitector, Poirot attributes 425 violation scenarios to 8 faults, cutting debugging time by 96.89% compared to manual analysis in practice.
Article Search Article: issta26main-p785-p (type: Full Paper) doi:10.1145/3832179

Deprecated but Not Abandoned: A Large-Scale Empirical Study on Growing-User-Demand Deprecated NPM Packages
Zezhou Tang, Yang Zhang, Xinjun Mao, Tanghaoran Zhang, Changrong Xie, Wenyu Xu, Simeng Yao, and Yiwen Wu
(National University of Defense Technology, China)
Package deprecation in ecosystems like NPM signals the termination of maintenance, and continued use of such packages poses potential sustainability and security risks to dependent projects. We observe a counter-intuitive phenomenon among widely-used deprecated packages whose user demand continues to grow after deprecation; we define these as Growing-user-demand Deprecated NPM Packages (GDNPs). Despite this clear contradiction between deprecation and growing user demand, the community engagement, reasons, and challenges of GDNPs have not been systematically examined. To bridge this gap, we conduct a mixed-method empirical study that identifies and analyzes 864 GDNPs from 4,011 widely-used deprecated packages, alongside surveys of 76 maintainers and 67 users. We find that GDNPs grow on average by 14.5% per month after deprecation, yet repository-level community engagement eventually drops significantly, revealing an expanding maintenance gap. Quantitatively, GDNPs contribute to over 124 million monthly exposures to high-severity vulnerabilities. Surveys indicate that continued reliance stems primarily from the complexity of the dependency tree and user inertia, leading to reactive maintenance and the accumulation of technical debt. Furthermore, topic modeling of post-deprecation discussions of GDNP repositories shows that community discussions heavily prioritize functional errors while seldom discussing security vulnerabilities, highlighting a misalignment between perceived and actual risk. Based on the results, we provide actionable implications that can facilitate future research and assist stakeholders in improving the maintenance of GDNPs.
Article Search Article: issta26main-p793-p (type: Full Paper) doi:10.1145/3832180

Generating Project-Specific Test Cases with Requirement Validation Intention
Binhang Qi, Yun Lin, Xinyi Weng, Yuhuan Huang, Chenyan Liu, Hailong Sun, Zhi Jin, and Jin Song Dong
(National University of Singapore, Singapore; Shanghai Jiao Tong University, China; Beihang University, China; Wuhan University, China; Peking University, China)
Test cases are valuable assets for maintaining software quality. State-of-the-art automated test generation techniques typically focus on maximizing program branch coverage or translating focal methods into test code. However, in contrast to branch coverage or code-to-test translation, practical tests are written out of the need to validate whether a requirement has been fulfilled. Specifically, a test usually reflects a developer’s validation intention for a particular scenario of a program function, regarding (1) what is the test scenario of a program function? and (2) what is the expected behavior under such a scenario? Without taking such intention into account, generated tests are less likely to be adopted in practice.
In this work, we propose IntentionTest, which generates project-specific tests given the description of validation intention. The design is motivated by two insights: (1) rationale insight: the description of validation intention regarding scenario description and behavioral expectation, compared to coverage and focal code, carries more crucial information about what to test; and (2) technical insight: practical test code exhibits high duplication, indicating that existing tests are highly reusable for how to test. Therefore, IntentionTest adopts a retrieval-and-edit manner. First, given a focal code and a description of validation intention consisting of a test objective with test precondition and expected results, IntentionTest retrieves a reusable test in the project as the test reference. Then, IntentionTest edits the test reference with an LLM regarding the validation intention toward the target test. To help the target test include a project-specific test prefix and a relevant assertion, IntentionTest further explores the software project to identify crucial code facts (i.e., relevant API/code to call and global variables to refer to in the test) as important context for the test generation. We extensively evaluate IntentionTest against four baselines (TELPA, DA, ChatTester, and EvoSuite) on 3,680 test cases from 12 open-source projects. Compared to state-of-the-art baselines, with a given validation intention, IntentionTest can (1) generate tests far more semantically relevant to ground-truth tests by (i) achieving common mutation scores 28.1% to 37.6% higher and (ii) achieving common coverage ratios 16.9% to 23.9% higher; and (2) achieve successful-pass rates 23.7% to 49.0% higher.
Article Search Article: issta26main-p802-p (type: Full Paper) doi:10.1145/3832181

Self-Adaptive Code Representation: Grammar-Guided or Token-Based?
Qingyuan Liang, Zhao Zhang, Chen Liu, Zeyu Sun, Zheng Lin, Yueyi Xiao, Haotian Zhang, Yingfei Xiong, and Lu Zhang
(Peking University, China; Institute of Software at Chinese Academy of Sciences, China; Kuaishou Technology, China)
Recent advances in code intelligence are largely driven by large language models (LLMs), which represent code as token sequences and achieve strong results on many software engineering tasks. However, standard token-based approaches often struggle to capture the inherent structural regularities of programming languages. While grammar-guided approaches can alleviate this limitation by modeling syntax more explicitly, they may sacrifice flexibility. In this work, we propose the Self-Adaptive Code Representation (SACR) framework, which enables models to dynamically select between token-based and grammar-guided representations depending on the problem context. SACR operates in two stages: first, the model is jointly trained on both representations to learn their respective strengths; second, an adaptive mechanism quantifies the generation difficulty for each sample and selects the model’s preferred code representation for training. We implement SACR in three versions of SacrCoder with parameter sizes of 0.5B, 1.3B, and 3B, and evaluate their effectiveness across a range of code generation and code repair benchmarks, including two repository-level pragmatic benchmarks built from real open-source projects. Through empirical analysis, we first demonstrate that token-based and grammar-guided code representations exhibit complementary strengths. Building on this observation, SACR selectively exploits these complementary representations during training to improve overall performance. Experimental results show that SacrCoder consistently achieves state-of-the-art performance. For example, the 3B model obtains a Pass@1 of 84.1% on HumanEval and 30.3% on APPS, corresponding to relative improvements of 9.5% and 32.3% over grammar-only and dual-format baselines, respectively, and SacrCoder also outperforms all single-representation baselines on the real-world programming benchmarks at every scale. Further ablation studies confirm that the effectiveness of SACR in leveraging complementary code representations.
Article Search Article: issta26main-p844-p (type: Full Paper) doi:10.1145/3832182

Test Case Prioritization for DNNs via Neural Collapse Instability
Chunyu Liu, Mingyuan Li, Yang Li, Wenmin Li, Fei Gao, Tengfei Tu, and Su-Juan Qin
(Beijing University of Posts and Telecommunications, China)
With the widespread deployment of deep neural networks (DNNs) in safety-critical domains, reducing the cost of model validation under limited testing budgets has become increasingly important. Existing test case prioritization techniques often rely on single-checkpoint confidence signals derived from output probabilities. However, DNNs can be confidently wrong, and the confidence margin between the predicted and competing classes is frequently small, which weakens early fault discovery. To address this limitation, we propose a Neural-Collapse-Inspired Prioritization (NCIP) framework that replaces absolute confidence with cross-checkpoint prediction variability in the terminal training regime, where model geometry becomes highly structured. NCIP introduces two key components. First, it selects an NC-guided representative subset of training checkpoints using an equiangularity score of classifier weights, quantified as the standard deviation of pairwise cosine similarities among class weight vectors. Second, it prioritizes test inputs by their prediction variability across the selected checkpoints, surfacing boundary-adjacent and failure-prone samples that are unstable under checkpoint-induced decision boundary shifts. Extensive experiments across multiple datasets and architectures show that NCIP achieves strong performance in early fault discovery compared with competitive baselines, with 1.5%–16.6% RAUC-ALL gains and 4.9%–20.6% RAUC-500 gains under the same testing budget. NCIP further attains the best average performance across all dataset-model pairs.
Article Search Artifacts Available Article: issta26main-p847-p (type: Full Paper) doi:10.1145/3832183

Reproduction Package for Article "Test Case Prioritization for DNNs via Neural Collapse Instability" (doi:10.5281/zenodo.21292329): This artifact provides the source code, experiment scripts, environment configuration, trained model checkpoints, and documentation for reproducing the main results of the paper "Test Case Prioritization for DNNs via Neural Collapse Instability." It implements the NCIP framework, including neural-collapse-guided ...

Evaluating the Impact of Explainable AI on Trust in AI-Assisted Code Review
Zhenhan Gao, Marvin Muñoz Barón, Umm-e Habiba, Daniel Graziotin, and Stefan Wagner
(Technical University of Munich, Germany; University of Hohenheim, Germany)
Background: The use of large language models (LLMs) for automated code review has brought significant change to a time-consuming part of software engineering. Prior work has shown that LLM-based code tools can improve code quality and enable more robust software development processes. As the tools get more powerful, the explanations behind their decisions remain hard to understand. Developers struggle to assess the validity of LLM-generated code reviews, making it difficult to gauge how much trust they should place in them. While the application of automated code review with LLMs has been extensively investigated, the inclusion of Explainable AI (XAI) for transparency in code reviews and its impact on trust are yet to be explored. Objective: We aim to address this research gap by studying the influence of XAI on the trust of software developers in AI-assisted code reviews. Method: We conducted a within-subjects user study with 34 participants from diverse programming backgrounds, comparing three experimental LLM-based automated code review systems with varying levels of XAI support: Condition A (detailed explanation and review feedback), Condition B (review feedback only), and Condition C (no explanations). Participants were shown a series of real-world code change requests along with the AI-generated code reviews. During the study, we measured trust perceptions for each system using a questionnaire, agreement with the AI recommendation, the reasoning for accepting or rejecting the code change, and the time taken to review the code change. Results: Our quantitative results show that the level of explanation significantly influences both the level of trust of software developers and their agreement with AI recommendations, but in different ways. Full explanations (Condition A) yield the highest perceived trust (M = 3.99/5) but not the highest agreement with AI recommendations, whereas moderate explanations (Condition B) achieve the highest agreement with AI (89.22%). This could suggest that more explanations prompt developers to question AI recommendations more frequently. In contrast, providing no explanations (Condition C) results in the lowest levels of trust and agreement. We also find that the level of explanation did not significantly impact the time taken to accept or reject a code change. Across all conditions, the most commonly cited reasons for code change decisions were changes in code readability and the correctness of the implementation. Conclusion: Overall, these findings indicate that incorporating XAI into the code review process significantly changes the trust perceptions and agreement with AI recommendations for software developers. These results provide insights for the design and evaluation of trustworthy AI-based code review systems, and support researchers in the design of studies on the human factors of AI-assisted software development.
Article Search Artifacts Available Article: issta26main-p848-p (type: Full Paper) doi:10.1145/3832184

Evaluating the Impact of Explainable AI on Trust in AI-Assisted Code Review - Data, Code and Diagrams (doi:10.5281/zenodo.21457282): Release version of the data, analysis code, user study data, instruments, diagrams used in and generated by our empirical study to ensure reproducibility, repeatability, and transparency, to be published alongside our paper “Evaluating the Impact of Explainable AI on Trust in AI-Assisted Code Review”.

RESTOR: Automated Test Oracle Generation for RESTful APIs via Reinforcement Learning
Xun Zhou, Zhen Dong, Mingyu Ren, Qiang Li, JunJie Li, Sifan Wang, Xiaolong Yu, Chaofeng Sha, and Xin Peng
(Fudan University, China; ByteDance, China)
Modern REST API testing faces a critical challenge in defining reliable test oracles, particularly in agile industrial environments where formal specifications (e.g., OpenAPI) are frequently missing or outdated, and historical execution logs are unavailable for newly deployed endpoints. In this paper, we present RESTOR (Reinforcement Enhanced Single-Traffic Oracle generator for REST APIs), a framework that generates executable test assertions from a single observed request-response pair in a black-box setting. Unlike existing approaches that rely on rule-based templates or massive training logs, RESTOR utilizes a novel data augmentation pipeline to fine-tune a lightweight Large Language Model (LLM) via Group Relative Policy Optimization (GRPO). This training process enables the model to internalize testing ”common sense” by optimizing a reward function that jointly encourages: (i) the selection of stable, semantically meaningful fields for validation and the avoidance of dynamic noise (e.g., timestamps or trace IDs); (ii) the generation of robust assertions that withstand logic variations. We evaluate RESTOR on an industrial dataset comprising over 2,300 API traces across 246 real-world services. Comprehensive experiments demonstrate that RESTOR significantly outperforms prompt-engineered baselines and generalist models, achieving a superior F₁ score of 85.42% in key field identification and increasing the proportion of semantically accurate assertions. Furthermore, deployment in a production CI/CD workflow at ByteDance confirms its practical value: the system raised the adoption rate of automatically generated test cases from 74.1% to over 96%, substantially reducing manual Quality Assurance (QA) effort while ensuring high execution stability.
Article Search Article: issta26main-p852-p (type: Full Paper) doi:10.1145/3832185

“You Are Deceived in the Pocket”: An Exploratory Study of Intrusive Advertisements in Mobile Applications
Miaoying Cai, Dongsun Kim, Lingling Fan, Xiangyu Zhang, and Sen Chen
(Nankai University, China; Korea University, Republic of Korea; Zhongguancun Academy, China)
Mobile advertising has become the primary monetization module for the Android ecosystem. However, this growth is accompanied by increasingly complex intrusive advertisements that undermine user agency through sophisticated behavioral interference. Current research on intrusive advertising primarily targets web or in-browser environments, failing to address the unique in-app characteristics of mobile intrusive ads. Existing ad analysis tools struggle to distinguish voluntary human actions from forced interactions due to the absence of intent-aware modeling. Furthermore, existing marketplace policies and legal frameworks lack unified terminology and enforceable rules, leading to inconsistent oversight. In this paper, we conduct an exploratory study to systematically investigate and model these intrusive advertising behaviors.
We propose a formal taxonomy grounded in regulatory policies, platform governance guidelines, industry standards, and documented real-world intrusive advertising practices from major SDK providers. Based on it, we apply an automated pipeline to detect intrusive patterns. Our analysis of more than 6,000 apps confirms the prevalence of mobile intrusive ads. Additionally, we perform a comparative analysis of mainstream regulations, uncovering significant misalignments in policy standards across different jurisdictions. Our study establishes a critical, theoretical, and practical foundation for ecosystem governance, enabling more effective detection and evidence-based policy refinement.
Article Search Artifacts Available Article: issta26main-p858-p (type: Full Paper) doi:10.1145/3832186

Artifact for “You Are Deceived in the Pocket”: An Exploratory Study of Intrusive Advertisements in Mobile Applications (doi:10.5281/zenodo.21834476): This artifact accompanies the ISSTA 2026 paper “You Are Deceived in the Pocket”: An Exploratory Study of Intrusive Advertisements in Mobile Applications. It provides the source code, data collection and preprocessing scripts, experimental scripts, analysis scripts, and supporting materials used in the study of ...

Is “Knowing It’s Malicious” Enough? Evaluating LLMs for Fine-Grained Malware Behavior Auditing
Xinran Zheng, Xingzhi Qian, Yiling He, Shuo Yang, and Lorenzo Cavallaro
(University College London, UK; University of Hong Kong, China)
Automated malware classifiers achieve strong detection performance, but auditing requires more than flagging a sample: analysts must explain malicious behaviors and justify them with concrete code evidence, a requirement that traditional signature-based methods and learning-based XAI often fail to satisfy in a human-interpretable manner. Large Language Models (LLMs) appear well-suited for this task due to their code reasoning and summarization ability, yet it remains unclear whether they can support reliable auditing. In particular, evaluating them faces three hurdles: (1) the lack of detailed, human-written behavior ground truth for reliable benchmarking; (2) real-world application codebases typically exceed the context limits of current models, which cannot be fully processed at once; and (3) the absence of reliable mechanisms to verify whether LLM-generated behavioral claims are faithfully supported by concrete code evidence.
Together, these obstacles make benchmarking LLM-based auditing non-trivial, leaving their true capabilities and failure modes opaque. To bridge this gap, we introduce MalEval, a diagnostic evaluation framework for systematically measuring the capability boundaries of LLMs in malware auditing. We pair real-world application codebases with expert-written audit reports to obtain fine-grained, behavior-level ground truth. Large codebases are compressed into unified behavior-relevant program contexts via a context-driven intermediate representation that preserves essential call relations. Both expert reports and model outputs are then mapped, through constrained reasoning, into structured evidence chains linking code-level facts to high-level behaviors in a common, comparable space. Built on this foundation, MalEval decomposes auditing into 4 stage-wise auditing tasks, allowing each intermediate judgment to be independently verified under limited context windows. We leverage MalEval to evaluate seven widely used LLMs and uncover clear capability boundaries: models rely on surface cues over verifiable evidence, struggle to compose dispersed facts into coherent attack chains, and are highly sensitive to context formulation. These findings shift the focus from optimizing isolated outputs to designing LLM and agentic workflows that can reliably support malware auditing.
Article Search Artifacts Available Article: issta26main-p865-p (type: Full Paper) doi:10.1145/3832187

Reproduction artifact of article Is "Knowing It’s Malicious" Enough? Evaluating LLMs for Fine-Grained Malware Behavior Auditing (doi:10.5281/zenodo.21833513): MalEval is a framework for evaluating Android malware behavior reports generated by large language models. The code (MalEval-main) in this repository implements two execution paths: From APK: run static analysis on APK files to produce entry points, call chains, function bodies, and reachable functions. From artifact: ...

Answer Is Cheap, Show Me the Evidence! Augmenting Automated Vulnerability Assessment with Evidence
Shengyi Pan, Zelong Zheng, Jiayuan Zhou, Xing Hu, Xin Xia, and Shanping Li
(Zhejiang University, China; Huawei, Canada; Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security, China)
Software Vulnerability (SV) assessment is a vital phase in SV management, which characterizes discovered SVs to locate hot spots and prioritize their remediation. To reduce the overhead and latency of manual assessment, prior works have explored automatically predicting assessment results from SV reports (SVRs). However, existing approaches fail to process the information conveyed by the rich text content (e.g., screenshots and code snippets) embedded in SVRs and miss information about vulnerable projects. More importantly, they primarily focus on assessment accuracy while neglecting to provide explanations or evidence supporting their predictions. As a result, these approaches remain impractical in real-world settings, where imperfect accuracy necessitates manual validation. LLMs offer a promising opportunity to address this limitation by performing SV assessment while simultaneously providing supporting evidence. Nevertheless, our extensive evaluation reveals that mainstream LLMs perform poorly on SV assessment tasks, largely due to a lack of assessment specific knowledge. To address the above challenges, we propose EAVA, a novel framework that effectively leverages LLMs to perform SV assessment and provide supporting evidence. EAVA employs specialized LLM agents to process rich text content in SVRs and incorporate information about vulnerable projects. EAVA builds a dedicated assessment LLM by injecting assessment-specific knowledge through finetuning. Specifically, we enable large-scale reasoning trajectory annotation using off-the-shelf LLMs and adopt a two-stage training paradigm, i.e., supervised instruction tuning to inject domain knowledge, followed by reinforcement learning to enhance the model’s intrinsic reasoning capability. Evaluations on a newly collected SVR dataset demonstrate that EAVA outperforms the best-performing baseline by 5.3%-35.2% across multiple evaluation metrics. Ablation studies validate the effectiveness of our design choices for both assessment specific model training and SV information enrichment. Finally, a user study with security experts confirms that the evidence provided by EAVA is useful and practical for real-world SV assessment.
Article Search Article: issta26main-p882-p (type: Full Paper) doi:10.1145/3832188

Bridging User Feedback and System Diagnosis: Reproducing Mobile Performance Issues from Reviews
Zhengquan Li, Zhenhao Li, Sidong Feng, Cuiyun Gao, Tao Zhang, and Zishuo Ding
(Hong Kong University of Science and Technology (Guangzhou), China; York University, Canada; Chinese University of Hong Kong, Shenzhen, China; Harbin Institute of Technology, Shenzhen, China; Macau University of Science and Technology, China)
Mobile application performance is a vital factor for user experience. Yet, performance issues are notoriously difficult to detect in development environments, where they often manifest less conspicuously, making their diagnosis more challenging. In this setting, app reviews from end users across diverse devices and usage contexts can provide timely and context-rich information about emerging performance issues. However, unlike structured bug reports, app reviews are written by end-users and tend to be more ambiguous, with individual reviews often providing only partial descriptions of the underlying issue. To bridge this gap, we present RevPerf, the first approach to automatically reproduce mobile application performance issues by leveraging and synthesizing information from app reviews. RevPerf retrieves complementary reviews via semantic retrieval and uses prompt engineering to integrate them, enriching the original review with performance issue details. An execution agent is then employed to generate and execute commands to reproduce the issue. After executing all necessary steps, the system incorporates multifaceted detection methods to identify performance issues by monitoring Android logs, GUI changes, and system resource utilization during the reproduction process. Experimental results demonstrate that our proposed framework achieves a 72.73% success rate in reproducing performance issues on the constructed dataset, outperforming the best baseline by 27.28 percentage points.
Article Search Artifacts Available Article: issta26main-p895-p (type: Full Paper) doi:10.1145/3832189

RevPerf (doi:10.6084/m9.figshare.29603171.v1): This artifact is the reproduction package for the article "Bridging User Feedback and System Diagnosis: Reproducing Mobile Performance Issues from Reviews". It contains the implementation of RevPerf, an approach that automatically reproduces mobile application performance issues from app reviews. RevPerf retrieves ...

Vbox: Efficient Black-Box Serializability Verification
Weihua Sun and Zhaonian Zou
(Harbin Institute of Technology, China)
Verifying the serializability of transaction histories is essential for assessing whether a database management system (DBMS) correctly enforces the claimed serializable isolation level. Black-box serializability verification provides a practical means for such a validation without relying on internal system details. Existing approaches often suffer from limitations, including incomplete anomaly detection, high verification overhead, excessive memory consumption, or dependence on specific concurrency control protocols. This paper presents Vbox, a black-box serializability verification method that incorporates support for predicate database operations, systematic use of transaction timing information, and a satisfiability (SAT)-based formulation with an efficient solver. Both theoretical analysis and experimental evaluation show that Vbox is correct and efficient, detects a wider range of data anomalies, and does not rely on any particular concurrency control protocol.
Article Search Article: issta26main-p898-p (type: Full Paper) doi:10.1145/3832190

Automated Dependency Optimization for Artifact-Based Build Systems
Hongxu Xu, Zhenyang Xu, Shane McIntosh, and Chengnian Sun
(University of Waterloo, Canada)
As projects grow, the maintenance of intra- and inter-project dependencies becomes increasingly complex. If dependency maintenance is lax, redundant dependencies may accrue, inflating incremental build and test latencies. The heterogeneity of language- and tool-specific dependency expressions and the complexity of the dependency graphs that they specify exacerbate the challenge of identifying and removing redundant dependencies. To address these challenges, this paper introduces DepReduce, an automated approach for optimizing declared dependencies in artifact-based build systems. DepReduce operates directly on the dependency graph managed by the underlying build system, and formalizes the optimization objective as minimizing the cumulative rebuild cost triggered by changes to individual targets. To achieve this, DepReduce performs the dependency lifting and dependency flattening operations on the dependency graph in topological order, which we prove is both correct and optimal under the defined optimization objective.
To empirically evaluate the approach, we implemented BazelDepReduce, an automated dependency optimization tool for Bazel. Bazel is an artifact-based build system with native support for multiple programming languages. We evaluated BazelDepReduce on 19 open-source Bazel projects written in seven programming languages. Among them, 16 projects across six languages achieved reductions in rebuild cost. In total, BazelDepReduce identified and removed 430 redundant dependencies, which we used to produce 16 Pull Requests (PRs). Twelve PRs have been merged by the target projects, including Angular and Apache RocketMQ, affecting up to 80.6% of subsequent commits, with a median of 26.3%. We also adapted BazelDepReduce to support Buck and Cargo, providing preliminary evidence that the implementation can be extended to other artifact-based build systems. Overall, these results show that our approach can effectively reduce rebuild cost on selected Bazel projects spanning multiple languages.
Article Search Artifacts Available Article: issta26main-p899-p (type: Full Paper) doi:10.1145/3832191

Automated Dependency Optimization for Artifact-Based Build Systems (Artifact) (doi:10.5281/zenodo.20670274): DepReduce is an artifact for the paper “Automated Dependency Optimization for Artifact-Based Build Systems.” It includes the DepReduce tool, example projects, experiment scripts, bundled evaluation data, and Docker images for reproducing the artifact-evaluation workflow. The artifact supports dependency optimization ...

Uncovering Business Logic Bugs via Semantics-Driven Unit Test Generation (Experience Paper)
Chen Yang and Junjie Chen
(Tianjin University, China)
Business logic bugs violate intended business semantics and are particularly prevalent in enterprise software. Yet most existing unit test generation techniques are code-centric, making such bugs difficult to expose. We present SeGa, a semantics-driven unit test generation technique for uncovering business logic bugs. SeGa constructs a semantic knowledge base from product requirement documents, represented as a set of functionality entries that group related requirements under a common business intent. Given a focal method, SeGa retrieves the relevant functionality entries and derives fine-grained business scenarios with explicit preconditions, triggering actions, expected outcomes, and semantic constraints to guide LLM-based test generation. We evaluate SeGa on four industrial Go projects containing 60 real-world business logic bugs. SeGa detects 22~25 more bugs than four state-of-the-art LLM-based techniques and improves precision by 26.9%~34.3%. Deployment across 6 production repositories further uncovers 16 previously unknown business logic bugs that were confirmed and fixed by developers, demonstrating SeGa's practical value. From our industrial study, we summarize a series of lessons and suggestions for practical use and future research.
Article Search Article: issta26main-p903-p (type: Full Paper) doi:10.1145/3832192

Just Old Wine in a New Bottle: Improving Vulnerability Repair Robustness to Combat Homologous but Heterogeneous Vulnerabilities
Xin Peng, Bo Lin, Yihao Qin, Jing Wang, Xin Zhao, Xiaoling Li, Jun Ma, Xiaoguang Mao, and Shangwen Wang
(National University of Defense Technology, China; Information Support Force Engineering University, China)
Collaborative development practices, such as forking and branching, enable parallel development and rapid iteration but significantly complicate vulnerability management. A major challenge is the propagation of homologous but heterogeneous vulnerabilities, where the same vulnerability appears within evolved code contexts that differ structurally and contextually across divergent branches or forks.Existing Automated Vulnerability Repair (AVR) approaches often fail in these scenarios due to their limited ability to generalize across structural and contextual variations. To address this problem, we propose EvoRepair, a robust AVR framework tailored for code evolution contexts. Our approach leverages a Retrieval-Augmented Generation based Evolution Specialist to synthesize code evolution variants, enriching the training data with diverse code contexts. We then employ Chain-of-Thought reasoning and curriculum learning to train a Repair Specialist, enabling it to capture invariant repair logic during structural and contextual changes. We evaluate EvoRepair on two datasets representing cross-branch and cross-fork scenarios. Evaluation results demonstrate that EvoRepair successfully repaired 35.33% of the vulnerabilities on average in evolution scenarios, outperforming the state-of-the-art baseline Vul-R2 by 42.12%. Furthermore, EvoRepair also demonstrates superior generalizability in non-evolution scenarios, achieving state-of-the-art performance on the PrimeVul dataset by fixing 25.75% of the vulnerabilities, surpassing the best baseline by 21.75%.
Article Search Article: issta26main-p916-p (type: Full Paper) doi:10.1145/3832193

SpectraDL: A Historical Issue-Driven, Test Specification-Assisted Transfer Testing Approach for Deep Learning Frameworks via LLMs
Shifan Liu, Chang-ai Sun, Fulei Wu, and Wing Kwong Chan
(University of Science and Technology Beijing, China; City University of Hong Kong, China)
Deep learning (DL) frameworks provide diverse fundamental algorithmic units as operators, which are critical infrastructure for constructing various intelligent software. Since mainstream frameworks widely adopt the open-source development paradigm, bugs may recur across operators and even across frameworks. Recent studies leverage large language models (LLMs) and historical issues to generate cross-framework test cases. However, existing approaches still suffer from two limitations. First, their test cases have low fault detection capability because they mainly reuse inputs or contexts from historical issues without considering the underlying root causes. Second, an effective mechanism for determining the appropriate transfer scope within a target framework is lacking. To overcome these limitations, we propose SpectraDL, a historical issue-driven, test specification-assisted transfer testing approach for DL frameworks. SpectraDL first extracts rigorous test specifications for each operator from official documentation, and then extracts and transforms historical issues and associated pull requests into structured fault representations (i.e., bug patterns). SpectraDL uses a dual retrieval mechanism based on semantic intent and structural input-space features to transfer these bug patterns to related operators across frameworks. Experiments on four mainstream DL frameworks show that SpectraDL detected 125 previously unknown bugs, 107 of which developers confirmed. The results confirm that SpectraDL delivers a promising transfer testing approach for DL frameworks.
Article Search Article: issta26main-p919-p (type: Full Paper) doi:10.1145/3832194

BCaLLM: Call Graph-Guided Python Breaking Change Detection with Large Language Models
Wei Cheng, Chen Shen, Huan Zhang, Yuhan Wu, Jingyue Yang, and Wei Hu
(Nanjing University, China)
Software libraries frequently evolve, introducing breaking changes that disrupt client applications. Existing detection approaches primarily target static programming languages or focus on syntactic changes, leaving behavioral breaking changes in dynamic languages such as Python underexplored. This task is particularly challenging due to side effects and call relationships, two critical factors that implicitly alter API behaviors and propagate change impact across library APIs. To address these challenges, we propose a generalized taxonomy of function API breaking changes. Grounded in Hyrum’s Law, our taxonomy is defined from the client’s perspective of observable behaviors and unifies both syntactic and behavioral categories in a multi-label formulation. Furthermore, we present BCaLLM, a novel framework to detect fine-grained breaking changes in Python packages by leveraging call graphs and large language models (LLMs). BCaLLM constructs a fused call graph to scope change impact, prunes compatible APIs and code context via memory-based heuristics, and employs an LLM to detect specific breaking changes. We construct PyBCEval, a manually annotated benchmark of 588 APIs from 27 version pairs of 19 widely used Python packages. Experiments with diverse LLMs show that BCaLLM outperforms text-based baselines by 3.71%–10.16% and LLM-based baselines by 1.60%–4.83% in F1-score.
Article Search Article: issta26main-p926-p (type: Full Paper) doi:10.1145/3832195

Compiling Large Multi-modal Requirement Documents into Runnable Software Systems: From an Agentic Test-Driven Perspective
Weiyu Kong, Yun Lin, Xiwen Teoh, Duc-Minh Nguyen, Ruofei Ren, Jiaxin Chang, Haoxu Hu, and Haoyu Chen
(Shanghai Jiao Tong University, China; National University of Singapore, Singapore)
Large Language Models (LLMs) have significantly improved programming efficiency by parsing natural language into code snippets. However, their performance degrades significantly as requirements scale; when faced with multi-modal documents containing hundreds of scenarios, LLMs often produce incorrect implementations or omit crucial constraints. Observing LLMs' ever-evolving capability and their persistent stochastic hallucination, we raise a question: whether it is possible to make LLM-based agentic programming go beyond "code generation" to "requirement compilation", i.e., whether programmers can produce a runnable system by only accomplishing (non-trivial) requirement documents?
In this work, we take a first step by proposing the ARC (Agentic Requirement Compilation) technique to parse a multi-modal requirement document, describing hundreds of scenarios in a DSL format, into a runnable software system. In addition to the source code, ARC also generates software engineering artifacts including (1) a modular design that spans the user interface, API interface, and database, (2) enriched test cases for each interface (including unit tests, modular tests, and integration tests), and (3) detailed traceability across all artifacts for software maintenance. Our approach employs a bidirectional test-driven agentic loop: (1) a top-down architecture phase that decomposes requirements into UI, API, and database interfaces, each of which is equipped with verifiable test suites, and (2) a bottom-up implementation phase where agents generate code that must satisfy the generated tests. Throughout this process, ARC maintains strict traceability across requirements, design, and code to facilitate intelligent asset reuse and follow-up maintenance. We evaluate ARC on two complementary benchmarks, i.e., a depth-oriented benchmark of 6 runnable web systems spanning 50-200 requirement scenarios, and the breadth-oriented AppForge benchmark comprising 101 Android app generation tasks. Across 3 independent trials, ARC outperforms all state-of-the-art LLM-based baselines, with the generated web systems passing on average 50.6% more GUI tests, and achieving 100% compile success and 68.3% test case pass rate on AppForge. In addition, a user study with 21 participants shows that participants with limited programming experience successfully write DSL-based documents consisting of 50 to 174 scenarios, within 5.6 hours on average, to generate a runnable system such as a real-world ticket-booking system of around 10K lines of code with maintainable architecture.
Article Search Article: issta26main-p928-p (type: Full Paper) doi:10.1145/3832196

COEUR: COhesion and Exhaustiveness of User Stories Representations
Marius Ortega, Hassan Imhah, Nédra Mellouli, Christophe Rodrigues, and Nicolas Travers
(De Vinci Higher Education, France; Onepoint, France)
User Stories are key artifacts in Requirement and Software Engineering. Despite their wide adoption, their writing in industrial contexts tends to diverge from the principles initially stated in Agile methodologies. In this context, sets of metrics such as INVEST or QUS emerged to qualify these items. In this paper, we argue that the said sets of metrics are only partially efficient at capturing the quality of user stories contextualized in a project, and that their actual adoption in business contexts is limited due to multiple aspects: their unfitness to specific contexts, the difficulty of implementation requiring human intervention, the absence of reproducibility or their misalignment with actual quality of user stories. Such limitations, prevent practitioners from efficiently applying them for downstream tasks such as LLM-based user story generation. To address these challenges, we introduce COEUR, a framework comprising two metrics: Cohesion and Exhaustiveness. These metrics are designed to mirror core Product Owner responsibilities. Specifically, Cohesion evaluates the structural organization and logical grouping of the backlog, while Exhaustiveness monitors the semantic alignment between the elicited needs and the proposed technical solutions. Additionally, they are quantitative, reproducible, and automatically computable. To evaluate COEUR, we conduct four empirical experiments organized into two validation tracks. The first track utilizes two noising-based experiments to assess our metrics' sensitivity to requirement degradation. The second track evaluates their performance in generative contexts through two standard learning paradigms for LLMs: In-Context Learning (ICL) and Supervised Fine-Tuning (SFT), both applied to our user story generation task. Subsequently, COEUR provides a turnkey measurement of Product Backlogs' quality for both project monitoring by human experts and LLM benchmarking applied to automatic user stories generation.
Article Search Artifacts Available Article: issta26main-p945-p (type: Full Paper) doi:10.1145/3832197

COEUR: COhesion and Exhaustiveness of User stories Representations (Artifact) (doi:10.5281/zenodo.21280932): The artifact repository contains a Docker image with: - The core package for COEUR (the main contribution of the paper) - The experiments presented in the paper - additional visualizations

Ghosts in the Memory: Detecting Unintended Sensitive Data in Android Apps
Seonghyeon Song, Taeyoung Kim, Woojoo Kim, Seojin Park, Sungjae Hwang, and Hyoungshick Kim
(Sungkyunkwan University, Republic of Korea)
Android apps often retain sensitive information such as co-users’ data, authentication tokens, and encryption keys in memory, leaving them vulnerable to unauthorized access. Existing dynamic analysis tools struggle to detect such memory-resident leaks because of three fundamental challenges: (i) runtime application self-protection (RASP) and other anti-analysis mechanisms actively detect and block conventional instrumentation;(ii) the semantic gap between high-level data types and their raw memory representations obscures sensitive content; and (iii) the transient nature of memory-resident data causes snapshot-based methods to misscritical exposures. We present Android-MRI, an OS-level analysis tool that embeds tracing directly into the Android kernel to detect unintended sensitive data leakage through memory while remaining invisible to application-layer defenses. Android-MRI provides instruction-granular monitoring to track the propagation of sensitive data in memory, surfacing the full context of unintended persistence. In an evaluation of 50 popular apps (≥1M installs), Android-MRI bypassed 97.6% of RASP protections (vs. 52.4% for Frida) and revealed 17 previously unknown memory disclosures. These included leaks of profile PINs, subscription-only content, and cryptographic keys. 12 vendors confirmed the issues; 4 have patched them, and 3 (Netflix, Smule, and Delivery Express) acknowledged them via bug bounty programs.
Article Search Article: issta26main-p953-p (type: Full Paper) doi:10.1145/3832198

PAGENT: Program Analysis Guided LLM Agent for Proof-of-Concept Generation
Achintya Desai, Md Shafiuzzaman, Wenbo Guo, and Tevfik Bultan
(University of California at Santa Barbara, USA)
Software developers frequently receive vulnerability reports that require them to reproduce the vulnerability in a reliable manner by generating a proof-of-concept (PoC) input that triggers it. Given the source code for a software project and a specific code location for a potential vulnerability, automatically generating a PoC for the given vulnerability has been a challenging research problem. Symbolic execution and fuzzing techniques require expert guidance and manual steps and face scalability challenges for PoC generation. Although recent advances in LLMs have increased the level of automation and scalability, the success rate of PoC generation with LLMs remains quite low. In this paper, we present a novel approach called Program Analysis Guided proof of concept generation agENT (PAGENT) that is scalable and significantly improves the success rate of LLM-based automated PoC generation compared to prior results. PAGENT integrates lightweight and rule-based static analysis phases for providing static analysis guidance and sanitizer-based profiling and coverage information for providing dynamic analysis guidance with a PoC generation agent. Our experiments demonstrate that the resulting hybrid approach significantly outperforms the prior top-performing agentic approach by 132% for the PoC generation task across 10 open-source projects. PAGENT also discovered 32 post-patch PoCs that trigger the vulnerability in the patched version of the source code, with 2 reproducing the crash in the most recent versions.
Article Search Artifacts Available Article: issta26main-p955-p (type: Full Paper) doi:10.1145/3832199

PAGENT artifact (doi:10.5281/zenodo.21057878): PAGENT automatically generates Proof-of-Concept (PoC) exploits for C software vulnerabilities by combining **static analysis**, **dynamic (coverage) analysis**, and an **iterative LLM agent**. It was evaluated on **203 vulnerabilities across 10 open-source C projects** drawn from the ARVO benchmark, achieving a ...

Steering Tree-of-Thought Reasoning via Deductive Verification
Haoliang Cheng, Enyi Tang, Shuoxiao Zhang, Jiahe Mao, Yanling Fu, Jason Ma, Keyu Cui, Yuchuan Liu, Yu Tian, Xinyu Gao, and Haibin Wang
(Nanjing University, China; Qingying Feifan, China)
Large language models (LLMs) have demonstrated great potential in code reasoning tasks, but their reasoning processes lack reliable verification mechanisms, making it difficult to ensure logical correctness. The Tree of Thoughts (ToT) framework improves reasoning by exploring multiple paths and employing backtracking, yet its path selection relies entirely on LLM-based self-evaluation—a heuristic and error-prone mechanism—leading to frequent erroneous pruning and unproductive exploration.
We identify a key insight: LLMs’ encoding capability is stronger than their reasoning capability—translating code semantics into formal constraints is a pattern-matching task that LLMs can perform reliably, while verification should be delegated to SMT(Satisfiability Modulo Theories) solvers. Based on this insight, we propose Deductive Steering, a mechanism that integrates SMT solver verification into Tree of Thoughts exploration. It consists of four core components: (1) Candidate Generator produces candidate reasoning steps, each comprising a natural language thought t and its SMT constraint encoding ϕ; (2) Deductive Evaluator verifies whether a candidate constraint ϕ is a logical consequence of the accumulated constraint Φ by checking the unsatisfiability of Φ ∧ ¬ ϕ; (3) Counterexample Refinement uses counterexample to guide the LLM in correcting its reasoning when verification fails; (4) Exploration and Backtracking Strategy manages path exploration and backtracks to alternative candidates when verification fails. Experiments on five benchmarks covering fault localization, program synthesis, and loop invariant generation show that, compared with ToT, Deductive Steering improves task-level effectiveness by 9.2–32.6 percentage points while reducing token consumption by 35.7–52.4%. The method generalizes across different LLMs and extends to mathematical reasoning, demonstrating broad applicability to domains where reasoning can be encoded as formal constraints.
Article Search Artifacts Available Article: issta26main-p972-p (type: Full Paper) doi:10.1145/3832200

Artifact for "Steering Tree-of-Thought Reasoning via Deductive Verification" (doi:10.5281/zenodo.21031775): Artifact for "Steering Tree-of-Thought Reasoning via Deductive Verification"

HELO-APR: Enhancing Low-Resource Program Repair through Cross-Lingual Knowledge Transfer
Zhipeng Wang, Boyang Yang, Yidong Wan, Liuye Guo, You Lv, Tao Zheng, Zhuowei Wang, and Tieke He
(Nanjing University, China; Yanshan University, China; Jiangsu University, China; Guangdong University of Technology, China)
Large Language Models (LLMs) perform well on automatic program repair (APR) for high-resource programming languages (HRPLs), but their effectiveness drops sharply in low-resource programming languages (LRPLs) due to the lack of sufficient verified buggy–fixed pairs for APR training. To address this challenge, we propose HELO-APR (High-resource Enabled LOw-resource APR), a two-stage APR framework that enables cross-lingual transfer of repair knowledge from HRPLs to LRPLs. HELO-APR (1) constructs high-quality LRPL training data by synthesizing LRPL buggy–fixed pairs from their HRPL counterparts, preserving defect-type consistency while ensuring that the synthesized code is idiomatic; and (2) adopts a curriculum learning strategy that progressively performs HRPL repair learning, cross-lingual repair alignment, and LRPL repair adaptation, thereby improving repair effectiveness in LRPLs. Using C++ as the source HRPL and Ruby and Rust as the target LRPLs, experiments on xCodeEval show that HELO-APR achieves the best macro-average Pass@k results and outperforms strong baselines in most settings. It increases Pass@1 from 31.17% to 48.65% on DeepSeek-Coder-6.7B and from 1.67% to 11.97% on CodeLlama-7B, while improving syntactic validity by raising the macro-average target compilation rate on CodeLlama from 49.77% to 91.98%. On Defects4Ruby, HELO-APR increases BLEU-4 from 61.20 to 66.79 and ROUGE-1 from 76.76 to 83.59 on CodeLlama-7B, indicating higher similarity to developer patches in real-world settings. Finally, we conduct ablation studies to assess the necessity of each core component. These results suggest that verified cross-lingual supervision provides a reusable approach for improving LLM-based repair in low-resource programming languages.
Article Search Article: issta26main-p975-p (type: Full Paper) doi:10.1145/3832201

On Behavioral Alignment of Model-Code and Human-Code Understandability via Behavioral Proxies
Xiaokai Rong, Aashish Yadavally, Hridya Dhulipala, Anh H. N. Nguyen, and Tien N. Nguyen
(University of Texas at Dallas, USA; University of Central Florida, USA)
Code understandability is a critical aspect of software quality. Prior research has largely focused on this attribute from a human-centric or code-centric perspective, while it should be viewed as a relational property arising from the interaction between a reader and the code. With the increasing adoption of large language models (LLMs) in software engineering, we posit that the notion of “reader” should be generalized to encompass both humans and models. Building on this relational perspective, we extend the concept of code understandability to distinguish between human and model code understandability, aiming to investigate the behavioral alignment between the two. To this end, we use a dataset from a prior study containing human-rated judgments of understandability across diverse participant groups, and evaluate multiple open and closed-source LLMs. To operationalize such behavioral alignment, we introduce four behavioral proxies of model-code understandability (BPMU): P0, based on program comprehension-focused question answering; and P1–P3, based on program intent summarization (derived from our proposed semantic self-consistency). Our findings show that LLMs exhibit stronger behavioral alignment with general human code understandability than previously used shallow machine learning baselines. Stratified analyses further reveal that model code understandability aligns most closely with professional developers, but significantly less with other student groups. Role-conditioned prompting does not improve the alignment for the latter, suggesting that current LLMs cannot accurately mimic the perspectives of human readers with varying expertise. Finally, we demonstrate that semantic self-consistency is a reliable and extensible measure to be used as a behavioral proxy for quantifying model code understandability, with broad implications in both software engineering research and practice.
Article Search Article: issta26main-p985-p (type: Full Paper) doi:10.1145/3832202

FuncDroid: Towards Inter-functional Flows for Comprehensive Mobile App GUI Testing
Jinlong He, Changwei Xia, Binru Huang, Jiwei Yan, Jun Yan, and Jian Zhang
(Institute of Software at Chinese Academy of Sciences, China)
As mobile application (app) functionalities grow increasingly complex and their iterations accelerate, ensuring high reliability presents significant challenges. While functionality-oriented GUI testing has attracted growing research attention, existing approaches largely overlook interactions across functionalities, making them ineffective at uncovering deep bugs hidden in inter-functional behaviors. To fill this gap, we first design a Functional Flow Graph (FFG), a behavioral model that explicitly captures an app’s functional units and their inter-functional interactions. Based on the FFG, we further introduce an inter-functional-flow-oriented GUI testing approach with the dual goals of precise model construction and deep bug detection. This approach is realized through a long–short-term-view-guided testing process. By combining two complementary test-generation views, it can adaptively refine functional boundaries and systematically explore inter-functional flows under diverse triggering conditions. We implement our approach in a tool called FuncDroid, and evaluate it on two benchmarks: (1) a widely‑used open‑source benchmark with 50 reproducible crash bugs and (2) a diverse set of 52 popular commercial apps. Experimental results demonstrate that FuncDroid significantly outperforms state‑of‑the‑art baselines in both coverage (+28%) and bug detection number (+107%). Moreover, FuncDroid successfully uncovers 18 previously unknown non‑crash functional bugs in commercial apps, confirming its practical effectiveness.
Article Search Article: issta26main-p994-p (type: Full Paper) doi:10.1145/3832203

Evaluating and Mitigating the Misguidance Effect of Buggy Code in LLM-Generated Unit Tests
Junda Zhao, Shurui Zhou, and Eldan Cohen
(University of Toronto, Canada)
While Large Language Models (LLMs) show great promise for automating unit test generation, recent studies suggest that the quality of generated tests can be negatively impacted when models are prompted with buggy code. This paper presents a new metric to quantitatively measure the "misguidance effect," a phenomenon where buggy code steers LLMs toward generating tests that validate its erroneous behavior rather than expose it. Our analysis reveals that prompting LLMs with buggy code has a severe, twofold impact: it significantly increases "misguided tests" that assert incorrect behavior while simultaneously suppressing the generation of effective, bug-finding tests. We further corroborate this effect from a model-internal perspective, showing that buggy code skews LLMs' preference toward tests that assert the same erroneous behavior. To counter this, we introduce and validate a specification-based unit test generation paradigm that replaces the code under test in the prompt with an LLM-generated specification docstring. Our results show that this paradigm effectively reduces misguided tests while substantially increasing effective tests, improves multi-round, feedback-driven test generation pipelines, and remains applicable to both buggy and bug-free code. Overall, these results suggest that specification-based prompting is a promising strategy for mitigating misguidance from buggy code in LLM-generated unit tests.
Article Search Artifacts Available Article: issta26main-p997-p (type: Full Paper) doi:10.1145/3832204

Reproduction Package for Article 'Evaluating and Mitigating the Misguidance Effect of Buggy Code in LLM-Generated Unit Tests' (doi:10.5281/zenodo.21428156): This package contains the full pipeline used in the paper to (1) quantitatively measure the "misguidance effect" — the phenomenon where prompting an LLM with buggy code steers it toward generating tests that assert the buggy behavior instead of exposing it — and (2) mitigate it via specification-based test generation, ...

Fuzzing the Boundary between Models and Code in Hybrid AI-Enabled Systems
Xinyu Gao, Yang Feng, Yuchen Lu, Zhenqian Liu, Zhenyu Chen, and Baowen Xu
(Nanjing University, China)
Deep learning (DL) techniques are increasingly integrated into traditional software systems, giving rise to hybrid AI-enabled systems that combine neural models with program logic. While these systems exhibit remarkable capabilities, their complex and heterogeneous architectures pose significant challenges for reliability and testing, particularly in safety-critical domains such as autonomous driving. Existing testing approaches either target traditional code or isolate neural networks, overlooking failures arising from their interactions.
In this paper, we present Neude, a lightweight and extensible coverage-guided fuzzing framework specifically designed for hybrid AI-enabled systems. Unlike existing tools, Neude combines observations of program execution and neural model coverage to guide input mutations toward unexplored state spaces, enabling systematic testing of the entire hybrid system. Moreover, Neude employs domain-aware mutation operators coupled with metamorphic relations, allowing automated bug detection without manual assertions. We evaluate Neude on Pylot, a complex autonomous driving system with tightly coupled neural and program components. Experimental results show that Neude uncovers diverse errors, and further analysis reveals how model uncertainty propagates through deterministic program logic to trigger downstream module failures. Our findings highlight the fragility of current hybrid architectures, calling for a paradigm shift from model-centric testing to system-centric quality assurance that accounts for the intricate interplay between neural and procedural components.
Article Search Article: issta26main-p1003-p (type: Full Paper) doi:10.1145/3832205

Unpacking AI Agent Participation in Issue-Centered Collaboration in Open-Source Software Development
Kaiwen Zhi, Guisheng Fan, and Wentao Chen
(East China University of Science and Technology, China; Shanghai Engineering Research Center of Smart Energy, China)
The increasing participation of AI agents in open-source software development raises questions about their role in collaborative processes. This paper investigates how AI agent participation relates to the structure and outcomes of issue-centered collaboration in open-source software projects. We adopt a process-level perspective by modeling issue handling as sequences of events and extending dynamic issue-pr entropy to distinguish between agent-related and non-agent-related contributions. Using large-scale issue event data from 83 GitHub repositories, we construct a project–month panel dataset and analyze associations between collaboration complexity and development outcomes. Our results show that agent-related collaboration complexity is more strongly associated with development output than human-only collaboration complexity, and is associated with fewer newly introduced defects. In contrast, its relationship with issue resolution efficiency is highly context-dependent. These findings highlight the importance of considering collaboration structure when evaluating the impact of AI agents in open-source software development.
Article Search Article: issta26main-p1013-p (type: Full Paper) doi:10.1145/3832206

Checked-In Secret Detection: Strings Are All You Need
Zhengdong Huang, Kevin Li, Jinqiu Yang, Yepang Liu, and Lili Wei
(Southern University of Science and Technology, China; McGill University, Canada; Concordia University, Canada)
Hardcoded secrets in source code pose critical security vulnerabilities which can be easily exploited by malicious adversaries. Existing regex-based detection approaches suffer from fundamental limitations, as secrets often lack identifiable patterns, resulting in poor precision and recall. Recent studies have explored context-aware detection methods, as surrounding code can reveal the purpose of candidate strings. However, these methods confront three key challenges: (1) obfuscation robustness where models over-rely on easily obfuscated identifiers, (2) cross-language generalization difficulties due to uneven training data distribution, and (3) lengthy and noisy context that introduces excessive irrelevant tokens and slows inference. We observe that strings serve as a critical information source for code semantics, offering superior contextual density, obfuscation robustness, and language independence. Based on this insight, we propose StringGroup, a novel context extraction algorithm that mines strings surrounding potential secrets. By introducing a relatively simple modification to existing patterns that narrows the analysis specifically to string literals, the method achieves significant gains. With only 33.2% of the original context, it preserves over 80% of semantic information and significantly improves the signal-to-noise ratio for secret detection. We further design a context-aware secret detection tool, Secretron, based on StringGroup methods and Transformer model. Evaluation on the SecretBench dataset demonstrates high accuracy with 98.74% F1-score and strong robustness under obfuscation and cross-language scenarios, outperforming state-of-the-art LLM-based baselines. We deploy our tool in real-world environments and successfully detect 48 previously unknown secret keys from 26 applications, demonstrating the practical effectiveness of our approach.
Article Search Artifacts Available Article: issta26main-p1021-p (type: Full Paper) doi:10.1145/3832207

Source Code (doi:10.5281/zenodo.21930185): This artifact contains the source code accompanying the paper.

Testing Retrieval-Augmented Generation Systems with Chunk Coverage
Jinhan Kim, Samuele Pasini, and Paolo Tonella
(USI Lugano, Switzerland)
Retrieval-Augmented Generation (RAG)-based systems are increasingly deployed in high-stakes settings where correct behaviour depends not only on the language model but also on the retrieval component that selects external documents at inference time. While existing RAG evaluation metrics assess retrieval and generation quality on a per-query basis, typically relying on query-level test oracles such as reference answers or relevance annotations, they provide limited insight into whether a test suite adequately exercises the retrieval behaviour of the system as a whole. In this paper, we introduce Chunk Coverage (CC), an oracle-independent test adequacy criterion for testing the retrieval component of RAG systems. CC measures the fraction of corpus chunks that are retrieved at least once across a test suite, providing a structural view of which parts of the retrieval space have been exercised. We further show how CC can be used to guide test selection and generation by prioritising queries that expand coverage of previously unexercised retrieval regions. We evaluate CC on clinical and financial RAG system scenarios. CC-guided testing reaches 50% of attainable coverage 1.7x faster than random selection and 4.2x faster than redundancy-biased strategies. Moreover, CC improves fault detection effectiveness (APFD) by 10% to 25% over random, indicating earlier discovery of distinct retrieval faults. These results show that CC captures retrieval diversity relevant to effective testing without requiring test oracles.
Article Search Article: issta26main-p1023-p (type: Full Paper) doi:10.1145/3832208

Function Calling as a Flexible LLM Defense Add-On: Capability and Application Exploration
Zhenlan Ji, Daoyuan Wu, Wenxuan Wang, Pingchuan Ma, Shuai Wang, Lei Ma, and Juergen Rahmel
(Nara Institute of Science and Technology, Japan; Lingnan University, Hong Kong; Renmin University of China, China; Zhejiang University of Technology, China; Hong Kong University of Science and Technology, China; University of Tokyo, Japan; University of Alberta, Canada; HSBC, Hong Kong)
Large language models (LLMs) exhibit impressive capabilities but are susceptible to adversarial attacks that induce harmful outputs. Although various defenses have been proposed, their practicality is restricted by substantial runtime overhead or degraded model helpfulness. Moreover, LLM applications typically have diverse and evolving security requirements that cannot be fully anticipated during the design of static defenses. These limitations call for a flexible, low-overhead defense mechanism that can be easily customized to meet task-specific needs.
In this paper, we explore function calling (FC)—a built-in mechanism in modern LLMs for invoking custom tools—as a lightweight and adaptable defense add-on. We show that by defining functions representing malicious actions, LLMs equipped with FC can intercept harmful prompts by triggering these function calls instead of generating unsafe content. Extensive experiments across mainstream LLMs demonstrate that FC substantially improves defense effectiveness with minimal impact on model helpfulness. To further assess FC's practical utility, we also introduce DSPEC, a new dataset reflecting real-world LLM applications with specific defense requirements. Our evaluations on DSPEC show that FC substantially outperforms existing defenses in this realistic setting. Besides, we also explore the practical applications of FC in various scenarios, including universal defense frameworks and multi-agent systems, further demonstrating its versatility and effectiveness in enhancing LLM security.
Article Search Article: issta26main-p1032-p (type: Full Paper) doi:10.1145/3832209

From Custom Logic to APIs: Understanding and Recommending API Replacement Refactorings
Bridget Nyirongo, Yanjie Jiang, Yuxia Zhang, and Hui Liu
(Beijing Institute of Technology, China; Tianjin University, China)
Software refactoring is essential for maintaining code quality. However, API replacement refactoring, which replaces custom logic with API calls, remains underexplored. Existing refactoring tools provide limited support for detecting such opportunities because they rely on predefined templates and have difficulty capturing complex, multi-statement semantic equivalents. To address this limitation, we conduct the first empirical study of API replacement refactorings by mining 166,299 commits across six open-source Java projects and manually analyzing a curated subset of 1,800 commits, from which we identify 366 validated instances to characterize their scope, categories, and recurring patterns. Based on these insights, we propose AKIRA (Adaptive Knowledge Discovery and Retrieval), a hybrid framework that integrates pattern-deterministic heuristics with a refactoring-aware knowledge base to assess the practical feasibility of recommending API replacement refactorings. Our evaluation shows that AKIRA achieves 90% recall and 88% precision on a manually curated dataset. Furthermore, on the external RETIWA dataset, AKIRA significantly improves the state of the art by increasing recall from 21% to 81% and precision from 40% to 78%. These results demonstrate the effectiveness of combining static pattern matching with semantic reasoning to support the automation of recommending complex API replacement refactorings.
Article Search Article: issta26main-p1038-p (type: Full Paper) doi:10.1145/3832210

IterTestQ: Assembly-Level, Cross-Platform Testing of Quantum Computing Platforms
Matteo Paltenghi and Michael Pradel
(University of Stuttgart, Germany; CISPA Helmholtz Center for Information Security, Germany)
Quantum computing platforms are susceptible to quantum-specific bugs, such as incorrectly ordering qubits or incorrectly implementing quantum abstractions. These bugs are difficult to detect and require specialized expertise. The field faces challenges due to a fragmented landscape of platforms and rapid development cycles that often prioritize new features over thorough testing, severely hindering the reliability of quantum software. To address these challenges, we present IterTestQ, a novel cross-platform testing approach for quantum computing platforms. The key technical contribution is our novel ITE process, which generates equivalent quantum programs by iteratively (I)mporting them into platform-specific representations, (T)ransforming the program via optimizations and gate conversions, and (E)xporting the program again. To transfer programs across platforms and test cross-platform consistency, IterTestQ leverages QASM, an assembly-level representation supported by most platforms. The approach uses a crash oracle to detect failures during cross-platform transformations and an equivalence oracle to validate the semantic consistency of the generated assembly programs, which are expected to be equivalent by construction. We evaluate IterTestQ on widely-used quantum computing platforms, including Qiskit, PennyLane, Pytket, BQSKit, and Cirq, revealing 23 bugs, 17 of which are already confirmed or fixed. Our results also demonstrate that IterTestQ complements existing quantum fuzzers (covering tens of thousands of otherwise uncovered lines), is efficient (with 0.00089 seconds per generated program), and that the ITE process is crucial for its effectiveness.
Article Search Article: issta26main-p1044-p (type: Full Paper) doi:10.1145/3832211

Integrating Multiple Features for Weakly-Supervised False-Passing Products Detection in Software Product Lines
Tao Zhang, Yan Lei, Haoran Xia, Huan Xie, and Chunyan Liu
(Chongqing University, China; Southwest University, China)
Software Product Lines (SPL) enable the efficient development of configurable systems through feature modularization. However, the inherent configurability of software introduces significant challenges for fault localization within these systems. A key challenge among these is the problem of false-passing products, configurable products that contain faulty code yet coincidentally pass all their associated tests, thereby masking faults and misleading diagnosis efforts. To mitigate the negative impact of false-passing products. Supervised detection approaches are often impractical due to their reliance on complete labels, which are unavailable during early testing phases. To address this, we propose PULP, a label-agnostic detection approach that exploits the execution similarity between failing and false-passing products. PULP extracts five categories of features and employs a weakly-supervised learning algorithm to identify false-passing products without pre-labeled data. Evaluated on 823 buggy versions from six real-world SPL systems, PULP achieves superior detection performance, with best accuracy of 90.33% and precision of 94.93% for false-passing products and consistently enhances fault localization rankings after eliminating the negative impact of false-passing product. This method offers a practical tool for SPL testing and debugging in label-incomplete environments.
Article Search Article: issta26main-p1055-p (type: Full Paper) doi:10.1145/3832212

Applying System Call Filtering to Real-World Binaries (Experience Paper)
Soumyakant Priyadarshan and Seyedhamed Ghavamnia
(Bloomberg, USA)
System call filtering restricts applications to the system calls they require, but inferring accurate syscall sets for binary-only programs remains challenging. A central difficulty lies in recovering precise control-flow information from binaries: over-approximation leads to overly permissive syscall filters, while missed control-flow edges result in unsound policies. Although many techniques have been proposed to improve control-flow recovery in binaries, their practical impact on syscall inference remains poorly understood.
In this work, we conduct an empirical study of syscall inference from binaries using multiple off-the-shelf binary analysis tools. We evaluate how different control-flow refinement techniques affect inferred syscall sets in practice. Our experiments on real-world applications show that refinements targeting individual control-flow transfers (e.g., call-site and callee argument matching) substantially improve per-call target precision but often do not reduce the overall syscall set. In contrast, techniques that reduce the global set of address-taken functions---such as leveraging relocation information to accurately identify code pointers---yield the most significant syscall reductions. Finally, we identify a practical lower bound on syscall reduction achievable via sound static binary analysis alone, and show that further improvements are likely to require configuration or workload-aware specialization.
Article Search Artifacts Available Article: issta26main-p1107-p (type: Full Paper) doi:10.1145/3832213

Applying System Call Filtering to Real-World Binaries (Experience Paper) (doi:10.5281/zenodo.21045094): A framework for binary analysis, call graph generation, and system call extraction. Supports multiple analysis backends, automatic dependency resolution, and binary hardening with SECCOMP filters.

Towards More Realistic Assertion Generation under Mixed-Assertion Scenario
Hongyan Li, Kunpeng E, Weifeng Sun, Quanjun Zhang, and Meng Yan
(Chongqing University, China; Singapore Management University, Singapore; Nanjing University of Science and Technology, China)
Unit testing is essential for software quality assurance, where a test case typically consists of a test prefix and an oracle, expressed as assertions. In practice, crafting high-quality assertions is non-trivial and time-consuming, as it requires developers to reason carefully about program states and expected behaviors. While recent advances in Large Language Models (LLMs) have shown promise for automating assertion generation (AG), current AG methods often rely on two unrealistic assumptions: (1) the Single-Assertion Formulation (A1), which assumes tests contain only one assertion, and (2) the Known-Position Formulation (A2), which treats AG as a "fill-in-the-blanks" task with pre-defined insertion points. Despite being widely adopted, the realism and implications of these assumptions have not been systematically examined. This paper revisits AG under a realistic Mixed-Assertion Scenario, where tests may contain one or multiple assertions and insertion positions are unavailable at inference time. To examine A1, we first conduct a large-scale empirical study of 358,117 developer-written tests from 7,061 projects. The results show that multi-assertion tests are prevalent, accounting for 40.32% of all tests and appearing in 92.87% of projects. Through manual analysis, we derive a taxonomy comprising ten fine-grained assertion patterns, showing that assertions in multi-assertion tests are rarely independent checks (4.69%) and instead coordinate to validate a unified test objective. To examine A2, we remove ground-truth insertion cues and observe substantial performance degradation, with Exact Match dropping by 11.80%--23.18% overall. This suggests that position cues affect not only where assertions are placed, but also the quality of what to assert. Motivated by these findings, we propose DA-AG, a two-stage framework designed for the realistic Mixed-Assertion Scenario with unknown insertion positions. In the first stage, it predicts assertion insertion positions to construct an assertion skeleton with explicit insertion cues. In the second stage, it generates assertion content conditioned on the resulting skeleton and retrieved exemplar assertion sequences. Extensive experiments across 13 diverse LLMs show that DA-AG consistently outperforms the corresponding one-stage baselines, which directly generate the completed test from the focal method and raw test prefix. DA-AG improves Exact Match by 32.24%--78.08% and CodeBLEU by 2.80%--8.63%, increases real-bug detection on Defects4J by 4--37 exposed bugs and 5--21 unique exposed bugs, and further improves other execution-based metrics, including compilability, bug-finding quality, and mutation scores. Moreover, DA-AG outperforms closed-source LLMs evaluated in a prompt-only setting without task-specific fine-tuning (e.g., GPT-4o and Claude-3.5) in similarity-based quality and real-bug detection.
Article Search Article: issta26main-p1109-p (type: Full Paper) doi:10.1145/3832214

OptiMine: Scalable and Precise Code Optimization for Android Apps via LLM-Driven Semantic Analysis
Pengbo Du, Qiuping Yi, Liangzheng Zhang, and Hongliang Liang
(Beijing University of Posts and Telecommunications, China)
Abstract: As Android applications grow in scale, latent performance issues increasingly degrade user experience and business outcomes, yet systematically identifying optimization opportunities in large codebases remains challenging. We present OptiMine, a hybrid knowledge-to-code framework that integrates large language models (LLMs) with static program analysis to automatically uncover actionable performance optimizations. OptiMine systematically transforms unstructured expert knowledge from documents, commit diffs, and reports into structured Optimization Signatures, enabling reproducible and context-aware program reasoning. These signatures drive scalable candidate retrieval via declarative Datalog queries, while LLM-guided semantic validation performs precise applicability checking, side-effect analysis, and impact-aware ranking. We evaluate OptiMine on a public benchmark and a large industrial Android codebase. The results show that OptiMine achieves higher precision and broader coverage than heuristic- and pattern-based baselines, while scaling effectively to industrial settings. Overall, OptiMine enables reliable and scalable performance auditing, bridging expert knowledge and actionable performance improvements in real-world mobile systems.
Article Search Artifacts Available Article: issta26main-p1110-p (type: Full Paper) doi:10.1145/3832215

OptiMine: Scalable and Precise Code Optimization for Android Apps via LLM-Driven Semantic Analysis (doi:10.5281/zenodo.21782781): The repository includes:The executable program of OptiMine, the public IID-issues dataset used in our generalization experiments and detailed README files with step-by-step instructions to reproduce the main results.

A Dataset of Reproducible Flaky-Test Failures
Suzzana Rafi, Mahbub-Ul-Hoque Sumon, Md Erfan, Maruf Morshed Khan, August Shi, and Wing Lam
(George Mason University, USA; Bangladesh Election Commission, Bangladesh; University of Alabama, USA; Ministry of Finance, Bangladesh; University of Texas at Austin, USA)
Flaky tests pass and fail non-deterministically when run on the same version of code. Previous research proposed techniques to detect, debug, and repair different categories of flaky tests. However, reproducing the flaky-test failures remains a major challenge due to their inherent non-determinism. Reliably reproducing flaky-test failures is essential for helping both developers and automatic techniques to debug and repair flaky tests. Many datasets related to flaky tests exist to help researchers study them, but these datasets are often composed of disjoint information, where each dataset provides some unique information over the others, such as flaky tests of many different categories, failure logs of flaky tests, or flaky tests reported by developers vs. flaky tests found by automated tools. Furthermore, several of them are missing the key aspect of providing a means to reliably reproduce the flaky-test failures. In this work, we aim to create a dataset of flaky tests, where each test’s failure is reproducible and there is a comprehensive set of information for the flaky test.
Compared to prior flaky-test datasets, our dataset is the first to provide (1) an environment to compile the code for running the flaky tests, (2) scripts to run the tests to reproduce the flaky-test failures, (3) scripts to automatically apply flaky-test fixes and check that the test is no longer flaky, and (4) execution logs of the flaky test both passing and failing. We present ReproFlake, a dataset of 1115 flaky tests, spread across four different flaky-test categories. We also publish the guideline we developed to construct our dataset so others can contribute to this dataset by collecting the same information. We also study the categories of flaky tests in ReproFlake, the location and size of flaky-test fixes, and the code coverage of flaky tests before and after they are fixed. Our study highlights promising future flaky-test research directions, which our dataset helps enable.
Article Search Article: issta26main-p1117-p (type: Full Paper) doi:10.1145/3832216

MuMuTestUp: Mutation-Based Multi-agent Test Case Update
Dawei Tian, Jiakun Liu, Yun Peng, Yichen Zhang, Jianlei Chi, Jun Sun, and Xiaohong Su
(Harbin Institute of Technology, China; Chinese University of Hong Kong, China; Xidian University Hangzhou Institute of Technology, China; Singapore Management University, Singapore)
Modern software systems evolve rapidly under continuous integration and deployment (CI/CD) practices, in which tests act as critical gatekeepers of software quality. However, when substantial code changes are introduced, existing test cases may become obsolete, leading to compilation failures, erroneous test behaviors, or inadequate regression coverage. Such issues can disrupt CI/CD pipelines, degrade development productivity, and ultimately undermine overall software quality. Many efforts are devoted to designing automatic test case update methods to address these issues. The most recent approaches rely on large language models (LLMs) to iteratively refine test cases using execution feedback from compilation errors or coverage reports, and on context retrieved via exact-matching approaches. They also prioritize test executability and line coverage to quickly build executable, correct test cases from the original broken test cases. Despite their correctness, current approaches face three limitations: (1) they focus on executabilty but overlook the adequacy of test assertions, which lowers the capability of test cases to detect faults; (2) they utilize only coarse line coverage singals instead of specific information about uncovered lines and branches; (3) they use exact-matching context retrieval approaches, which fails to provide accurate context given potential hallucinated queries from LLMs.
To address these challenges, we propose MuMuTestUp, a Mutation-guided, Multi-agent framework for automated test case updating. MuMuTestUp integrates three specialized agents: (1) a Mutation Analysis agent that leverages surviving mutants as indicators of weak or missing test assertions and generates individual repair instructions to strengthen or synthesize assertions for each surviving mutant, (2) a Coverage Analysis agent generates individual repair instructions for each uncovered line, uncovered branch rather than exposing raw coverage signals to the LLM, and (3) a Semantic Retrieval agent that uses semantic-similarity search to handle unavailable or hallucinated symbols. Additionally, we construct Prbench, a pull-request–level dataset of 571 samples from 10 open-source Java projects that considered cross-commit update scenarios, validated through three rounds of execution following prior studies to detect outdated tests. We evaluate MuMuTestUp against state-of-the-art baselines using both open-source and closed-source LLMs (Deepseek-V3.2 and GPT-4.1). With GPT-4.1, MuMuTestUp achieves a line coverage of 88.94%, branch coverage of 63.36%, and mutation score of 72.39%, outperforming the best baseline by 5.33%, 19.93%, and 16.66%, respectively.
Article Search Article: issta26main-p1161-p (type: Full Paper) doi:10.1145/3832217

Names Are All You Need: Effective and Safe Regression Test Selection for Python
You Wang, Michael Pradel, and Zhongxin Liu
(Zhejiang University, China; CISPA Helmholtz Center for Information Security, Germany)
Regression test selection (RTS) reduces the cost of regression testing by executing only those tests affected by a code change. Despite extensive study of RTS in statically typed languages such as Java, achieving effective and safe RTS in Python is challenging. Python’s dynamic typing makes precise call-graph construction difficult, which can cause call-graph-based RTS to miss affected tests, and hence, compromise safety. Python’s eager importing mechanism, in contrast, renders file-level dependency analysis overly conservative. This paper presents NameRTS, the first Python RTS approach based on fine-grained dependency analysis. NameRTS models a Python program as a bipartite graph of code element nodes (e.g., classes, functions, global variables) and name nodes (i.e., identifiers used to reference code elements), with edges capturing definitions and references. RTS is formulated as a reachability problem on this graph: a test is selected if any modified code element is reachable from the names used in that test. This design avoids call-graph construction, enabling a conservative analysis amenable to safety. To control dependency cascades introduced by coarse name matching, NameRTS applies two pruning strategies that leverage prior test executions and context information to refine name matching. To evaluate NameRTS, we construct the first Python RTS dataset with a ground truth indicating which test files are affected by each commit. It includes 500 commits drawn from 10 real-world Python projects. We compare NameRTS with the best-performing baseline, BabelRTS, an RTS technique based on coarse file-level dependencies. On this benchmark, NameRTS skips 69.90% of test files on average, outperforming BabelRTS by 146.5%. It also reduces end-to-end testing time by 45.59%, yielding a 107.7% improvement over BabelRTS. In terms of safety, NameRTS selects all affected tests for 99.6% of commits, with only rare misses in exceptional cases. In contrast, BabelRTS is safe for 76.6% of commits. These results demonstrate the effectiveness of NameRTS, paving the way for more efficient regression testing in Python.
Article Search Artifacts Available Article: issta26main-p1184-p (type: Full Paper) doi:10.1145/3832218

Names Are All You Need: Effective and Safe Regression Test Selection for Python (Artifact) (doi:10.5281/zenodo.21279428): NameRTS is a fine-grained regression test selection (RTS) technique for Python. It uses name-based dependency propagation, models Python programs as a bipartite graph of code elements and names, and formulates test selection as a reachability problem over this graph. The artifact contains: the NameRTS implementation; ...

From Static to Dynamic: Benchmarking Real-World Code Review with MCR-Bench
Dewu Zheng, Yanlin Wang, Xiwen Wang, Kefeng Duan, Hongyu Zhang, Xilin Liu, Yuchi Ma, and Zibin Zheng
(Sun Yat-sen University, China; Chongqing University, China; Huawei Cloud, China)
In real-world software development, code review typically involves iterative interactions between developers and reviewers to improve software quality, making the process costly and time-consuming. Although recent work explores large language models (LLMs) for automated code review, most approaches oversimplify code review into a single-round, static decision task, which fails to capture the multi-round interactive nature and the complex problem-solving processes inherent in realistic review scenarios. To bridge this gap, we introduce MCR-bench, the first defect state-aware benchmark designed for realistic multi-round code review. MCR-bench covers five commonly-used programming languages and consists of 2,269 real-world multi-round code review tasks, each of which is annotated with fine-grained defect information and cross-round state labels. Each task in MCR-bench is equipped with fine-grained defect metadata (e.g., description, type, severity) alongside dynamic state annotations, capturing the complete evolutionary trajectory of a defect throughout the multi-round process. We obtain several findings through extensive experiments on MCR-bench with mainstream LLMs. (1) Limited overall capability: experiments reveal that mainstream LLMs exhibit limited overall performance in defect detection and defect lifecycle state tracking, with performance degrading significantly as the number of interaction rounds increases; (2) Defect-sensitive performance: LLMs' performance varies substantially across different defect types and severity levels, with semantically complex or low-salience defects being significantly more likely to be missed; (3) Underlying Failure Mechanisms: our in-depth error analysis dissects the distinct drivers of false positives and false negatives, revealing critical weaknesses such as cross-round temporal misalignment and inadequate long-range memory.
Article Search Article: issta26main-p1189-p (type: Full Paper) doi:10.1145/3832219

Rust’s Type Checker Implementation Is Unsound: An Empirical Study on Soundness Bugs in rustc
Yusung Sim, Sukyoung Ryu, and Jaemin Hong
(KAIST, Republic of Korea; UNIST, Republic of Korea; Yale University, USA)
Rust is claimed to be a type-sound language capable of preventing various undesirable behaviors, including memory bugs. However, rustc, the official Rust compiler, is not immune to defects; it contains soundness bugs, where the compiler accepts programs that should be rejected during type checking. In this work, we present an empirical study of 30 issues that report potential soundness bugs in rustc, collected from the GitHub issue tracker between January 1, 2022 and September 1, 2025. We analyze each issue in depth, focusing on its affected feature, symptom (how the feature is mishandled), consequence (the resulting undesirable behavior), triggering features, community consensus regarding whether it is a bug, and lifecycle, including introduction, discovery, and fix. Furthermore, we investigate existing artifacts, including implementations such as AddressSanitizer, Miri, Chalk, and a-mir-formality, alongside documentation such as the Rust Reference, the FLS, and Rust RFCs to assess their potential as oracles for testing the type soundness of rustc. Our key findings indicate that: (1) Certain soundness bugs, typically triggered by implied bounds or trait objects, compromise memory safety. (2) Sound type checking is challenged by edge cases involving associated types and the interaction between lifetimes and traits. (3) Most bugs persist from the initial introduction of the relevant features and require significant time to be discovered. (4) While AddressSanitizer and Miri can detect soundness bugs that lead to memory bugs, a-mir-formality and Chalk are currently immature despite their potential to identify other bug categories. (5) Existing documentation frequently fails to provide precise explanations of the language semantics.
Article Search Artifacts Available Article: issta26main-p1206-p (type: Full Paper) doi:10.1145/3832220

Rust's Type Checker Implementation is Unsound: An Empirical Study on Soundness Bugs in rustc (Artifact) (doi:10.5281/zenodo.20728406): This is the artifact for the paper "Rust's Type Checker Implementation is Unsound: An Empirical Study on Soundness Bugs in rustc" submitted to ISSTA'26. In the paper, we study 30 soundness issues collected from the rustc GitHub issue tracker, each representing a case where a developer reported that the compiler ...

Solving String Split Constraints via Structural Relaxation
Rui Han, Ziheng Wang, Baoquan Cui, Yuhang Dong, Fuqi Jia, Feifei Ma, and Jian Zhang
(University of Chinese Academy of Sciences, China; Institute of Software at Chinese Academy of Sciences, China; IUniversity of Chinese Academy of Sciences, China)
String operations are integral to program analysis, yet reasoning about the ubiquitous split operation remains a challenge. SMT solvers have difficulty with split because it transforms a string into a variable-length sequence, creating a structural mismatch that leads to uninterpreted abstractions or unsound bounded approximations. In this paper, we bridge this gap with a precise, SMT-LIB-compliant encoding. Our key insight is structural relaxation: exploiting the sparsity of real-world constraints, we decouple the split structure from strict length requirements, materializing segments only on demand. We further introduce position-aware constraints to handle complex regex-based delimiters without overlaps.
We evaluated our framework on 580 benchmarks using four leading string solvers. Our encoding enables off-the-shelf solvers to handle split constraints, solving 157 out of 168 real-world benchmarks and outperforming current baselines. Notably, our framework involves complex string operations, revealing 12 previously unknown implementation bugs in mainstream solvers.
Article Search Article: issta26main-p1223-p (type: Full Paper) doi:10.1145/3832221

Augmenting Multi-technique Static Analysis with Large Language Models: A Neuro-symbolic Approach to Smart Contract Vulnerability Detection
Junxiang Wang, Fu Song, Miaomiao Zhang, Bowen Du, and Rongcan Pei
(Tongji University, China; Institute of Software at Chinese Academy of Sciences, China; University of Chinese Academy of Sciences, China; Nanjing Institute of Software Technology, China)
Smart contracts facilitate and enforce agreements between untrusted parties without trusted intermediaries, but vulnerabilities within contracts can cause severe damage once exploited. Various analysis techniques have been proposed for vulnerability detection, but they are typically limited to specific vulnerability types. While recent frameworks employ aggregation to broaden detection capacity, they often accumulate false positives due to loose integration that fails to resolve underlying conflicts. Meanwhile, the semantic reasoning capacity of large language models (LLMs) has shown promise in detecting vulnerabilities, despite inherent reasoning bottlenecks and hallucinations. Recognizing these challenges, in this work, we propose a novel neuro-symbolic approach, named Ensemble LLM-Assisted Static Analysis (ELSA). ELSA comprises two key modules, namely, LLM-assisted static analysis and analyzer ensemble (the ensemble of multiple LLM-assisted static analyzers), each incorporating two distinct strategies. The LLM-assisted static analysis augments individual analysis techniques with constraint-guided neural semantic reasoning, while the analyzer ensemble resolves conflicting outputs to distill a robust consensus. We evaluate ELSA on a comprehensive benchmark, including three open-source datasets and additional self-constructed Zero-Knowledge Proof-based smart contracts whose complexity poses unique challenges to static analysis. Experimental results demonstrate that our approach achieves an overall improvement of at least 17% over baselines and advanced mainstream approaches, effectively bridging semantic gaps and synergizing the complementary advantages of different analyzers. Furthermore, an ablation study and fine-grained analysis are conducted to investigate the key factors contributing to overall performance gains.
Article Search Artifacts Available Article: issta26main-p1237-p (type: Full Paper) doi:10.1145/3832222

Appendix to "Augmenting Multi-Technique Static Analysis with Large Language Models: A Neuro-Symbolic Approach to Smart Contract Vulnerability Detection": Supplementary appendix containing additional details and materials for the paper.

ELSA: Ensemble LLM-Assisted Static Analysis (Artifact) (doi:10.5281/zenodo.21896899): ELSA (Ensemble LLM-Assisted Static Analysis) is a neuro-symbolic approach for comprehensive smart contracts vulnerability detection that synergizes multiple static analysis techniques enhanced by Large Language Models (LLMs). By leveraging the unique strengths of different static analyzers and the semantic reasoning ...

Rethinking Mixture-of-Experts for Vulnerability Detection: An Empirical Study and Improved Design
Rongze Jiang, Chaofeng Sha, and Xin Peng
(Fudan University, China)
Deep learning–based vulnerability detection (DLVD) has recently adopted the Mixture-of-Experts (MoE) paradigm to address vulnerability heterogeneity and the long-tailed distribution of Common Weakness Enumeration (CWE) categories. A representative framework, MoEVD, builds CWE-specific experts and a router for expert selection, but its assumptions about expert specialization and routing reliability remain underexplored. We reproduce MoEVD under the same dataset, splits, backbone, and evaluation protocol, and conduct a fine-grained empirical study of expert behavior and routing decisions. We find that CWE-based experts do not consistently develop stable or exclusive specialization, and their effectiveness is highly sensitive to non-target vulnerabilities. More importantly, an idealized-router baseline shows that routing mismatch measurably limits what the original experts can achieve under idealized routing, while learned routing disproportionately sends non-vulnerable samples to a few experts. Guided by these findings, we explore targeted changes that relax strict CWE-based expert binding and replace fixed top-k routing with probability-mass-based top-p selection under a controlled OR-style voting rule. Under a controlled OR-style comparison on BigVul, the combined design shifts the precision--recall trade-off, improving F1 from 0.38 to 0.42 and recall from 0.32 to 0.39 while slightly increasing FPR. Repeated runs and zero-shot checks on two external datasets suggest that the trend is stable in the evaluated settings, while the improvements remain modest.
Article Search Article: issta26main-p1249-p (type: Full Paper) doi:10.1145/3832223

RecurIC3: Exploiting Structural Lemma Reuse to Accelerate IC3
Yuhan Li, Liangze Yin, Xinyi Gong, Minghao Liu, Tun Li, Wei Dong, and Ji Wang
(National University of Defense Technology, China)
IC3/PDR has become a widely adopted technique for safety model checking due to its high efficiency. Despite its success, the algorithm often suffers from redundant exploration due to the lack of a cross-level memory mechanism. This results in the repetitive discovery of highly similar CTIs (Counterexamples to Induction), forcing the solver to waste computational effort traversing overlapping blocking chains. We propose RecurIC3, a framework that alleviates this bottleneck via structural reuse. RecurIC3 maintains a Bad State Tree (G_bad) that persistently records CTIs together with their level-aligned predecessor–successor links along blocking chains, turning the blocking phase into a history-aware process. To reduce solver calls, RecurIC3 first retrieves and rechecks lightweight candidates from G_bad and falls back to solver queries only when reuse is exhausted. This approach can significantly reduce the search space, thereby enhancing the verification efficiency of IC3. We implemented RecurIC3 in the state-of-the-art model checker Kind2 and evaluated it on the official benchmark suite. On instances where reuse is triggered, RecurIC3 reduces the number of explored tree nodes by 27%, achieves a 1.42× cumulative speedup, and solves 16 additional instances (13 Safe and 3 Unsafe) within the same timeout. These results suggest that structural reuse can substantially accelerate IC3.
Article Search Article: issta26main-p1263-p (type: Full Paper) doi:10.1145/3832224

CausalRepair: Bridging the Causality Gap in Large Language Model-Based Automated Program Repair via Dual-Slicing
Linhao Wu, Yizhou Chen, Zhen Yang, Pengyu Xue, and Dan Hao
(Peking University, China; Shandong University, China; Hong Kong Polytechnic University, China)
Automated Program Repair (APR) aims to automatically fix buggy programs. In recent years, with the rapid advancement of Large Language Models (LLMs), LLM-based APR techniques have achieved significant progress. Despite their potential, the effectiveness of LLMs relies heavily on the quality of the provided repair context. However, existing LLM-based APR approaches suffer from a causality gap when constructing such contexts. Specifically, on the test side, existing methods struggle with test context ambiguity arising from noise interference or dependency absence; meanwhile, on the source side, existing retrieval-augmented methods primarily rely on static analysis inevitably introduce static over-approximation, resulting in contexts filled with unexecuted code and noise. Consequently, these contexts mislead LLMs, hindering them from identifying the true root cause and leading to incorrect fixes.
To bridge this gap, we introduce the concept of minimal causal context, defined as the essential set of dependencies required to explain a specific failure. Based on this, we propose CausalRepair, a novel conversation-driven APR framework that instantiates this concept through a synergistic dual-slicing strategy. Specifically, CausalRepair employs context-aware static slicing on the test side to purify test semantics, and utilizes execution-trace-based dynamic slicing on the source side to capture precise runtime dependencies. This constructs a high-quality context causally relevant to the bug, which filters out irrelevant code and guides the iterative repair process. We evaluate CausalRepair on the widely used Defects4J (V1.2 and V2.0) and the latest Defects4J-Trans benchmarks. To ensure a fair comparison, we unify the backbone model as DeepSeek-V3 in all experiments. The results demonstrate that CausalRepair correctly fixes 313 bugs on Defects4J, significantly outperforming state-of-the-art approaches such as ReinFix and TSAPR, while reducing the average repair cost to $0.029 per bug, achieving a dual optimization of effectiveness and efficiency.
Article Search Artifacts Available Article: issta26main-p1266-p (type: Full Paper) doi:10.1145/3832225

Reproduction Package for Article "CausalRepair: Bridging the Causality Gap in Large Language Model-Based Automated Program Repair via Dual-Slicing" (doi:10.5281/zenodo.20953899): This artifact is a self-contained Docker image that packages the CausalRepair framework, its dataset metadata, and the correct patches produced in our study. CausalRepair is an LLM-based Automated Program Repair (APR) framework that bridges the "causality gap" via a dual-slicing strategy: it constructs a Minimal ...

Towards Understanding the Bugs in Verilator, a Hardware Description Language Compiler
Songyan Jiang, Maolin Sun, Kang Chen, Qingyang Li, Yibiao Yang, and Yuming Zhou
(Nanjing University, China)
Verilator is the premier open-source Hardware Description Language (HDL) compiler. It transforms Verilog and SystemVerilog designs into optimized C++ or SystemC models, enabling high-speed, cycle-accurate simulation prior to large-scale production. As a cornerstone of the hardware verification ecosystem, the correctness of Verilator is paramount; compiler faults can lead to silent simulation errors or unexpected failures, undermining the integrity of the hardware development lifecycle. Unlike traditional software compilers, HDL compilers manage unique concurrency and synthesis semantics, potentially introducing distinct bug patterns and complexities. However, while prior research has explored testing techniques for HDL toolchains, there remains a lack of systematic empirical studies characterizing the specific nature of bugs in Verilator. This knowledge gap hinders the development of targeted improvements in compiler robustness and testing strategies. To address this, we present the first comprehensive empirical study of Verilator bugs. We manually collected, analyzed, and categorized a dataset of 488 confirmed bugs from the official repository over three years. Our study investigates bug symptoms, root causes, and the characteristics of triggering test cases, while also evaluating the effectiveness of existing testing techniques. Based on our findings, we provide actionable guidance for developers to enhance Verilator's reliability and for researchers to design more effective automated testing methodologies for HDL compilers.
Article Search Article: issta26main-p1270-p (type: Full Paper) doi:10.1145/3832226

DeepSCA: Dependency-Aware Software Composition Analysis for C/C++ Based on a Curated Code Feature Database
Meiqiu Xu, Xibin Zhao, Wenxuan Yu, Zhiliang Zhu, Li Li, Heqing Huang, and Ying Wang
(Northeastern University, China; Beihang University, China; City University of Hong Kong, China)
Software Composition Analysis (SCA) identifies reused third-party libraries (TPLs) and their dependencies, forming the basis of Software Bills of Materials (SBOMs) for securing software supply chain. Current C/C++ SCA techniques extract code features and match them against a TPL feature database. However, detection recall is limited by feature database coverage, and two key challenges remain: (1) Coverage-Precision Trade-off: Feature databases built from a single repository (e.g., GitHub) often yield high false negatives, as C/C++ TPLs are inherently distributed across diverse hosting platforms (e.g., Debian, ArchLinux, and Xrepo), each offering partially overlapping yet distinct sets of TPLs. Although a multi-repository feature database improves TPL coverage, it inevitably introduces numerous shared functions (e.g., cross-TPL code clones) that map to a broader set of candidate origins. When coupled with the limitations of existing SCA techniques in tracing provenance, this ambiguity further increases false positives. (2) Neglected Dependency Context: Existing SCA techniques primarily focus on detecting reused TPLs but often overlook their inter-dependencies. However, license compatibility constraints are inherently determined by directed dependency relationships among TPLs.
DeepSCA addresses Challenge 1 by constructing a feature database with 66,143,596 functions extracted from 33,100 C/C++ TPLs, covering nine mainstream hosting repositories. To reduce false positives from shared functions, DeepSCA groups TPLs with shared code, and uses distinctiveness scores to trace functions to their origins—ensuring high precision without sacrificing recall. To overcome Challenge 2, DeepSCA generates the dependency graph from the detected TPLs. it resolves filename ambiguity by simulating compiler header search paths and mitigates symbol ambiguity via namespace-aware resolution, enabling accurate file-level dependency mapping to TPLs. Evaluated on a ground-truth dataset of 335 OpenHarmony projects (10,583 TPLs, 64,200 dependencies), DeepSCA achieves 85.6% precision and 87.3% recall for TPL detection, outperforming state-of-the-art tools (e.g., 16.8% precision and 21.1% recall improvement over TPLite). Its dependency graphs reach 91.8% precision, surpassing Cneps by 11.3%. Moreover, the dependency graph generated by DeepSCA uncovers previously hidden combinations of license incompatibility issues in real-world projects.
Article Search Article: issta26main-p1376-p (type: Full Paper) doi:10.1145/3832227

MalTotal: Cost-Effective and Language-Agnostic Malicious Code Poisoning Detection for Millions of Repositories
Jian Zhao, Shenao Wang, Qingyang Wu, Yanjie Zhao, Xiao Cheng, and Haoyu Wang
(Huazhong University of Science and Technology, China; Macquarie University, Australia)
The widespread adoption of open source software (OSS) has introduced significant security risks, with malicious code poisoning attacks increasingly targeting public package registries and open-source platforms. Existing detection approaches, including heuristic-, learning-, and LLM-based methods, suffer from language-specific designs, limited generalization, and high analysis costs, making them unsuitable for large-scale multi-language analysis. To address these challenges, we propose MalTotal, a scalable and cost-effective framework for language-agnostic malicious code detection. MalTotal leverages LLM-assisted semantic reasoning to identify sensitive APIs, perform hybrid semantic slicing, and reconstruct malicious behavior contexts while reducing analysis overhead. Our evaluations show that MalTotal outperforms 8 state-of-the-art baselines, achieving an average F1-score of 93.1
Article Search Article: issta26main-p1381-p (type: Full Paper) doi:10.1145/3832228

Repair-Driven Greybox Fuzzing
Bachir Bendrissou, Alastair F. Donaldson, and Cristian Cadar
(Imperial College London, UK)
We present repair-driven greybox fuzzing, a new approach to greybox fuzzing that combines the strengths of unstructured, byte-level input mutation and grammar-guided input generation. By increasing input diversity while preserving input validity, repair-driven fuzzing promises to improve bug-finding ability for systems under test such as programming language interpreters that consume highly structured inputs. Our idea is to first mutate an input using a standard byte-level mutator, typically leading to an invalid input, and then repair the input using a grammar. Aggressively breaking and then repairing an input provides an effective way to reach parts of the input space that would be left unexplored by both byte-level and idiomatic grammar-based mutations. We put this idea into practice via RepairFuzz, a new greybox fuzzer based on AFL++, leveraging the byte level mutations of AFL++ and using the CPCT+ error recovery algorithm for input repair. We present experiments applying RepairFuzz to six SUTs covering four programming language input formats (Lua, PHP, JavaScript and Ruby), and present an experimental comparison with AFL++, Grammarinator and Nautilus, the state-of-the-art in standard greybox fuzzing and grammar-guided fuzzing. Our evaluation shows that RepairFuzz was able to find 13 confirmed bugs that were previously unknown, including 9 that were not found by AFL++, Grammarinator or Nautilus. Further, RepairFuzz yields absolute increases in code coverage for several SUTs and substantial complementary code coverage across all.
Article Search Article: issta26main-p1385-p (type: Full Paper) doi:10.1145/3832229

CLIR: Liveness-Driven and Structure-Aware Fuzzing for the Cranelift Compiler
Shangtong Cao, Tianlei Song, Qiuping Yi, Tianyu Chen, Guoai Xu, Ningyu He, and Haoyu Wang
(Beijing University of Posts and Telecommunications, China; Harbin Institute of Technology, Shenzhen, China; Peking University, China; Hong Kong Polytechnic University, Hong Kong; Huazhong University of Science and Technology, China)
Modern compilers are complex software systems that must correctly translate high-level programming languages into machine code across multiple architectures. Cranelift, a fast and modern compiler backend originally developed for WebAssembly and recently adopted as an experimental backend for Rust, has gained increasing importance due to its superior compilation speed compared to LLVM and comprehensive multi-architecture support, including x86-64, AArch64, s390x, and RISCV64. However, despite decades of development in compiler testing, testing Cranelift still presents unique challenges, including (1) constructing valid IR under the strict enforcement of SSA form, (2) generating sequences with sufficient computational density to stress backend components, and (3) balancing broad backend coverage with efficient root cause analysis across heterogeneous architectures.
To address these challenges, we propose CLIR, a differential testing framework that integrates a syntax-preserving hierarchical generation strategy to guarantee SSA validity, a liveness-guided instruction refinement mechanism to maximize computational density, and a diagnosis-guided cross-architecture adaptation scheme to facilitate efficient root cause analysis across heterogeneous backends. Our comprehensive evaluation demonstrates that CLIR substantially outperforms existing state-of-the-art baselines, detecting 8×, 24×, and 8× as many unique bugs as cranelift-fuzzgen, wasm-smith, and WASMaker, respectively, while RustSmith uncovered no bugs. Within 72 hours of testing, CLIR discovered 24 bugs spanning all target architectures, with 21 confirmed and 9 fixed.
Article Search Artifacts Available Article: issta26main-p1387-p (type: Full Paper) doi:10.1145/3832230

Artifact for “CLIR: Liveness-Driven and Structure-Aware Fuzzing for the Cranelift Compiler” (doi:10.5281/zenodo.21695797): CLIR is the artifact for the paper “CLIR: Liveness-Driven and Structure-Aware Fuzzing for the Cranelift Compiler.” It generates executable Cranelift IR (CLIF) programs using liveness-aware instruction construction and structured control-flow generation. CLIR supports both target-specific generation and compatible ...

Testing Static Taint Analyzers with Equivalence Modulo Taint
Maria Christakis, Anastasia Isychev, Samuel Pilz, Florian Tesarek, and Valentin Wüstholz
(TU Wien, Austria; Consensys Diligence, Austria)
Static taint analyzers are widely used to detect security vulnerabilities, yet their complexity makes them prone to soundness and precision issues. Validating these analyzers is challenging because ground-truth taint flows are rarely available and differential testing requires multiple comparable tools. To address this challenge, we introduce Equivalence Modulo Taint (EMT), a testing oracle for static taint analysis that defines program equivalence in terms of preserved source-sink flows rather than program semantics. EMT enables testing a single analyzer without ground-truth labels by checking consistency of reported flows across equivalent-modulo-taint program variants. Based on EMT, we present TaintCC, a framework that generates equivalent-modulo-taint variants through semantically equivalent, taint-oblivious, and taint-aware transformations targeting recurring difficulty dimensions in taint analysis. We evaluate TaintCC on four widely used analyzers---FlowDroid, Mariana Trench, Pysa, and Semgrep---and uncover 16 unique developer-confirmed issues, showing that even mature analyzers, whether academic or industrial, remain susceptible to reliability issues.
Article Search Article: issta26main-p1389-p (type: Full Paper) doi:10.1145/3832231

How Does Killing Surviving Mutants Help Detect Real Bugs with Assertion Generation? A Controlled Experiment
Hang Du, Vijay Krishna Palepu, and James A. Jones
(University of California at Irvine, USA; Microsoft, USA)
Killing surviving mutants is a central activity of mutation testing. This activity is motivated by the coupling-effect hypothesis: tests that expose simple artificial faults can also detect more complex, previously unseen real bugs. Despite these claimed benefits, automated studies have not directly measured the causal impact of mutant killing on real-bug detection. This limitation stems from open-ended mutant-killing strategies and a fundamental evaluation asymmetry that obscures causal attribution. In this work, we present the first large-scale controlled experiment that directly measures whether killing surviving mutants, without knowledge of future real bugs, would have enabled their detection. We model mutant killing as a selective, incremental process under realistic budget constraints, and we restrict test improvements to assertion augmentation. This restriction enables precise attribution of each test augmentation to a specific mutant-killing action. To support the experiment, we design a fully automated, fault-based assertion-augmentation technique that operates uniformly on mutants and real bugs and integrate it into Defects4J. Our controlled experiment yields several key empirical insights: (1) Across 642 Defects4J bugs, we find that 104 bugs would become detectable by adding an additional assertion to an existing passing, non-triggering test. (2) When coupling exists, a real bug is, on average, coupled with 21 surviving mutants, through which mutant killing can produce triggering tests. This number is substantially higher than the average of two mutants reported in prior studies. In those studies, coupling is inferred solely from documented bug-fixing tests rather than from tests derived via mutant killing. (3) Among these bugs, 63 of the 104 are detectable through principled mutant-killing (test augmentation) process. Notably, killing a randomly selected 30% of relevant surviving mutants, using only one assertion per mutant, suffices to detect 84.5% of these bugs. (4) By substituting mutants with real bugs and comparing their resulting assertion augmentation outputs, we find that real bugs induce broader behavioral effects than mutants, affecting more memory state locations, variables, and tests. (5) When mutation-derived assertions detect real bugs, they validate program outputs that overlap with, and are often strict subsets of, those affected by the real bugs. This offers a mechanistic explanation for why killing simple mutants can enable the detection of more complex real bugs.
Article Search Article: issta26main-p1437-p (type: Full Paper) doi:10.1145/3832232

The Illusion of Success: Learning-Based Android Malware Detectors (Replicability Study)
Michael Tegegn and Julia Rubin
(University of British Columbia, Canada)
Since 2012, hundreds of machine-learning-based classification approaches have been proposed to help separate malware and benign Android applications. These approaches typically collect a large number of applications of both types, split them into training and testing subsets, train a binary classifier on the training subset, and measure accuracy on the testing subset. They typically report very high achieved accuracy, especially highlighting the capability to accurately detect malware samples, i.e., recall of around 90%. More recentwork highlighted several biases and flaws in the experimental setup and evaluation methodology of such approaches, questioning the trustworthiness of their reported results. In an effort to better understand the current status of malware detection, we first conduct a systematic literature review to extract the properties of existing tools and the datasets that they use. We then design a large-scale longitudinal study, evaluating the most prominent tools on systematically collected datasets of applications drawn from AndroZoo or VirusShare repositories, while controlling for the known biases in the experimental setup. Our results show substantial instability in the classification accuracy, e.g., the accuracy of a tool can range from around 60% to 90%, even across different datasets drawn from the same repository. These results not only challenge the claimed detection capabilities of the tools but also render comparisons of reported tool accuracy practically meaningless, as tool evaluations, even in the best case, often draw applications from the same data repository rather than use the exact same dataset. To further explore the detection abilities of the tools, we design a deliberately naïve and unreliable classifier, which uses application package names as features for classification, and show that it performs comparably and sometimes even better than the state-of-the-art tools when compared under the same setup. Our results demonstrate that the Android malware detection problem is still far from being solved and call for the creation of more reliable, semantic malware detection tools.
Article Search Article: issta26main-p1444-p (type: Full Paper) doi:10.1145/3832233

No One-Size-Fits-All: Adaptive Code Editing with Feature-Based Strategy Selection
Jun Wan, Zhongxin Liu, Dajun Chen, Wei Jiang, Yong Li, and Xiaoxue Ren
(Zhejiang University, China; Ant Group, China)
Large language models have shown strong potential in automating code editing (CE) tasks, yet most existing systems rely on a fixed CE strategy (i.e., one-step, search-replace, and plan-apply) across diverse scenarios. In practice, CE tasks vary widely in code context, edit locality, and semantic complexity, causing no single strategy to perform consistently well. Moreover, systematic studies on the relationship between strategies and CE tasks remain scarce, limiting further optimization of LLM-based code editing strategies. To address this gap, we construct CEBench, comprising 500 real-world CE tasks curated from SWE-bench. Then we compare three representative CE strategies (one-step, search-replace, plan-apply). Our study reveals two key findings: (1) each strategy exhibits distinct advantages for specific CE tasks, with no universal optimality; (2) strategy effectiveness is significantly influenced by four core task features: total lines of code, modification type, modification scope, and task type. Motivated by these findings, we propose Ace, an adaptive code editing framework that dynamically selects optimal CE strategies based on task features. Unlike classifier-based or instruction-prompting approaches, Ace employs a fine-tuned LLM that learns feature-strategy mappings from empirical data to recommend strategies with interpretable rationales. Extensive experiments on our CEBench-verified show that Ace outperforms state-of-the-art baselines: it boosts Pass@1 by 21.7% while reducing token consumption by 27.3% compared to the best baseline (AutoCodeRover). Our strategy selector achieves 72.6% selection accuracy with 69.9% rationale accuracy (Cohen’s Kappa=0.807), providing transparent and trustworthy decision-making for developers.
Article Search Article: issta26main-p1446-p (type: Full Paper) doi:10.1145/3832234

When Optimizations Backfire: The Paradox of Plaintext Optimizations in Privacy-Preserving ML Compilers
Yichen Li, Jin Tan, Dongwei Xiao, Yiteng Peng, Pingchuan Ma, Junming Ma, Shoumeng Yan, Shuai Wang, and Fengwei Zhang
(Southern University of Science and Technology, China; Ant Group, China; Hong Kong University of Science and Technology, Hong Kong, China; Zhejiang University of Technology, China; Hong Kong University of Science and Technology, China)
Growing concerns about data security and privacy have fueled the widespread adoption of Privacy-Preserving Machine Learning (PPML). Cryptography-based PPML, which allows computation directly on encrypted data, significantly mitigates data leakage risks. To facilitate its adoption, Ciphertext Machine Learning Compilers (CMLCs) automate the translation of high-level ML procedures into low-level circuits for encrypted data. Recently, CMLCs have increasingly adopted infrastructure from Plaintext Machine Learning Compilers (PMLCs). While integrating Plaintext-Domain Optimization Passes (PDOPs) into CMLCs offers potential performance, usability, and extensibility benefits, our study shows it can also lead to severe performance regressions — a risk that has been largely overlooked.
To address this, we introduce Hopta, a hybrid domain optimization defects tester and analyzer, aiming to find optimization bugs that can degrade the performance of compiled circuits from CMLCs. We carefully design two core components: (1) an optimization pipeline mutation mechanism to detect optimization anomalies by selectively enabling/disabling PDOPs, and (2) a profile-guided code reduction tool that efficiently simplifies defect-triggering programs to isolate optimization anomalies and facilitate debugging. Applying Hopta to SecretFlow-SPU, a production-grade CMLC, we identified 11 optimization defects (spanning matrix indexing,arithmetic/boolean conversion, and cost model deviation) that led to substantial performance regressions, with compiled circuits incurring up to 310.2% increased cost. Our comprehensive analysis provides empirical insights into fundamental differences between plaintext and ciphertext domain optimization strategies, offering crucial guidance for future CMLC development. We conclude with a brief discussion of extensions to other hybrid ML compilers, underscoring the methodology’s compiler-agnostic nature. This work establishes a new research direction for enhancing CMLC performance and the practical deployment of privacy-preserving ML systems.
Article Search Article: issta26main-p1486-p (type: Full Paper) doi:10.1145/3832235

Test Case Selection for Deep Neural Networks: A Replication Study on LLMs for Code (Replicability Study)
Ali Asgari, Mitchell Olsthoorn, and Annibale Panichella
(Delft University of Technology, Netherlands)
Test case selection (TCS) techniques have been explored to support the operational evaluation of deep neural networks (DNNs) under limited testing budgets, where labeling cost is a primary concern and early model failure detection is a key objective. Although prior studies report promising results, existing empirical evaluations focus almost exclusively on vision-based DNNs and datasets. As observed in recent surveys, models and datasets specifically designed for software engineering tasks have not been considered, leaving it unclear whether prior findings generalize to LLM code models.
This paper presents a large-scale replication study of TCS techniques in the context of LLM code models. We re-examine established TCS strategies originally proposed for DNNs and complement them with statistical sampling strategies that have not previously been evaluated for TCS. We assess their effectiveness on three code-related classification tasks: clone detection, vulnerability detection, and technical debt prediction. The study spans 17 task-specific fine-tuned model instances, 7 predictive features, and 13 selection strategies, including 12 feature-aware strategies and simple random sampling (SRS) as a feature-agnostic baseline. We evaluate performance along two dimensions: operational accuracy estimation and early failure discovery.
The results indicate that only a subset of findings reported for vision-based DNNs generalize when TCS is applied to LLMs for code. In particular, uncertainty-based features are effective for early failure discovery, while representation-based features are more robust for accuracy estimation. At the same time, performance varies substantially across tasks and models, indicating that the effectiveness of TCS techniques is context-dependent. Overall, this study provides empirical evidence on the replicability of TCS techniques beyond vision-based deep learning and offers insights into their use for the operational evaluation of LLMs for code.
Article Search Artifacts Available Article: issta26main-p1511-p (type: Full Paper) doi:10.1145/3832236

Test Case Selection for Deep Neural Networks: A Replication Study on LLMs for Code (Replicability Study) (doi:10.5281/zenodo.21336820): Replication package for the paper "Test Case Selection for Deep Neural Networks: A Replication Study on LLMs for Code (Replicability Study)" and accepted at the The ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) 2026

CAST: A Compiler-Based Framework for Systematically Testing LLM Compositional Safety
Lu Yan, Zhuo Zhang, Xiangzhe Xu, Shengwei An, Guangyu Shen, Zhou Xuan, Xuan Chen, and Xiangyu Zhang
(Purdue University, USA; Columbia University, USA; Virginia Tech, USA)
Large language models (LLMs) are increasingly used in software pipelines, raising concerns about harmful behaviors in security-critical domains. Existing safety evaluations predominantly probe models with single prompts or short interactions, and therefore do not capture how safety behaves under multi-step workflows where individual requests are composed into complex behavior. This paper introduces compositional safety, the property that an LLM remains safe not only against isolated malicious prompts, but also under structured, long-horizon decompositions of harmful intents. We propose CAST, a systematic testing framework designed to evaluate the compositional safety of LLMs in the domain of malicious code. Drawing inspiration from modern compiler infrastructures, CAST decouples test case generation from test execution using a novel intermediate representation, CAIR. This architecture allows the framework to automatically refine high-level testing intents into granular sub-tasks that serve as unit tests for the model’s alignment. These components are subsequently instantiated by the SUT and reassembled according to the CAIR control structure. The resulting artifact is then evaluated by intent-fulfillment scoring and, for the severity subset, external behavioral detectors and manual inspection. We evaluate CAST on four state-of-the-art LLMs across three security-critical testbeds. Our results demonstrate that CAST systematically exposes severe safety violations in strongly aligned models that resist conventional red-teaming, achieving up to a 365% increase in successful test cases compared to baseline testing strategies
Article Search Artifacts Available Article: issta26main-p1531-p (type: Full Paper) doi:10.1145/3832237

CAST: A Compiler-Based Framework for Systematically Testing LLM Compositional Safety (doi:10.5281/zenodo.21010284): Goal. This replication package is the artifact for the ISSTA 2026 paper "CAST: A Compiler-Based Framework for Systematically Testing LLM Compositional Safety." CAST evaluates whether a safety-aligned "System Under Test" language model can be driven, through a structured multi-step decomposition, to produce functional ...

CAM: A Causality-Based Analysis Framework for Multi-agent Code Generation Systems
Zongyi Lyu, Zhenlan Ji, Songqiang Chen, Liwen Wang, Yuheng Huang, Shuai Wang, and Shing-Chi Cheung
(Hong Kong University of Science and Technology, China; Guangzhou HKUST Fok Ying Tung Research Institute, China; Nara Institute of Science and Technology, Japan; Hong Kong University of Science and Technology, Hong Kong; University of Tokyo, Japan)
Despite the remarkable success that Multi-Agent Code Generation Systems (MACGS) have achieved, the inherent complexity of multi-agent architectures produces substantial volumes of intermediate outputs. To date, the individual importance of these intermediate outputs to the system correctness remains opaque, which impedes targeted optimization of MACGS designs. To address this challenge, we propose CAM, the first Causality-based Analysis framework for MACGS that systematically quantifies the contribution of different intermediate features to system correctness. By comprehensively categorizing intermediate outputs and systematically simulating realistic errors on intermediate features, we identify the important features for system correctness and aggregate their importance rankings, facilitating comprehensive analysis of MACGS.
We instantiate CAM on representative MACGS across multiple backend LLMs and datasets and conduct extensive empirical analysis on the identified importance rankings. Our analysis reveals intriguing findings: first, we uncover context-dependent features—features whose importance emerges mainly through interactions with other features, revealing that quality assurance for MACGS should move beyond module-level validation to incorporate cross-feature consistency checks; second, we reveal that hybrid backend MACGS with different backend LLMs assigned according to their relative strength achieves up to 7.3% Pass@1 improvement, underscoring hybrid architectures as a promising direction for future MACGS design. We further demonstrate CAM’s practical utility through two applications: (1) failure repair, which achieves a 73.6% success rate by optimizing top-3 importance-ranked features and (2) feature pruning, that reduces up to 33.6% intermediate token consumption with negligible or sometimes positive performance impact by pruning low-importance features. Our work provides actionable insights for MACGS design and deployment, establishing causality analysis as a powerful approach for understanding and improving MACGS.
Article Search Article: issta26main-p1561-p (type: Full Paper) doi:10.1145/3832238

KaPilot: LLM-Assisted Generation of Kani Specifications for Unsafe Rust Verification
Minghua Wang, Yuxi Ling, Mingzhi Gao, Yuwei Liu, and Lin Huang
(Ant Group, China; National University of Singapore, Singapore)
Rust’s ownership and type system provide strong memory safety guarantees, but unsafe code still presents memory safety risks. Formal verification is crucial for ensuring memory safety, but writing precise specifications for unsafe Rust is challenging and largely manual. Large language models (LLMs) have shown promise in generating formal specifications but are often code-centric, prone to inheriting implementation flaws, and lack systematic quality assessment.
In this paper, we present KaPilot, a multi-agent framework for automatically generating specifications to verify unsafe Rust memory safety using Kani. The process begins with lightweight program analysis and proof harness generation. The SafetyReq agent extracts a concise, refined list of safety requirements from the target Rust function’s documentation, which guides the SpecGen agent in producing initial specifications that specify memory safety concerns. Then, the specifications are iteratively refined through a generate–precheck–verify loop involving SpecGenerate, SpecPrecheck, and SpecVerify agents, which assess quality and feed errors back. By executing this loop multiple times, KaPilot generates a set of candidate specifications. Finally, the shuffle-and-implication strategy is applied to systematically determine the best specification from these candidates. We evaluated KaPilot on 54 unsafe Rust functions with ground truth and 44 without. KaPilot achieved 88.9% and 69.2% specification generation success, respectively, with 57.4% of generated specifications equivalent to or stronger than the ground truth. Compared with AutoSpec, KaPilot produces 14.8% more verifiable specifications and 25.9% more equivalent-or-better specifications.
Article Search Artifacts Available Article: issta26main-p1588-p (type: Full Paper) doi:10.1145/3832239

Reproduction Package (Docker container) for the ISSTA 2026 Paper "KaPilot: LLM-Assisted Generation of Kani Specifications for Unsafe Rust Verification" (doi:10.6084/m9.figshare.32834699.v3): This is the artifact for the paper "KaPilot: LLM-Assisted Generation of Kani Specifications for Unsafe Rust Verification" at ISSTA 2026. It presents an automated approach that leverages large language models to assist in generating specifications for unsafe Rust functions. We provide our artifact as a Docker ...

RippleGUItester: Change-Aware Exploratory Testing
Yanqi Su, Michael Pradel, and Chunyang Chen
(TU Munich, Germany; CISPA Helmholtz Center for Information Security, Germany)
Software systems evolve continuously through frequent code changes, yet such changes often introduce unintended bugs despite extensive testing and code review. Existing testing approaches are largely constrained to predefined execution paths or rely on unguided exploration, leaving many change-induced issues undetected. To address this challenge, we present RippleGUItester, a change-driven testing system that treats a code change as the epicenter of a ripple effect and explores its broader, user-visible impacts via the GUI. Given a code change, RippleGUItester performs LLM-based change-impact analysis to generate and enrich realistic test scenarios, executes these scenarios on both pre-change and post-change versions of the system, and applies differential analysis to identify behavioral differences. Crucially, RippleGUItester employs multimodal bug detection, comparing visual GUI changes and interpreting them in the context of natural-language change intents to distinguish unintended bugs from intended behavioral updates. We evaluate our approach on hundreds of real-world code changes across four widely used software systems: Firefox, Zettlr, JabRef, and Godot. Our results show that the proposed approach uncovers bugs introduced by code changes that were missed by existing test suites, CI pipelines, and code review. In total, we identify 26 previously unknown bugs that still exist in the latest versions of the evaluated systems. After reporting, 18 bugs have been fixed, 3 have been confirmed, and 5 were marked as intended. We envision RippleGUItester being applied before or shortly after a code change is merged, enabling earlier detection of regressions.
Article Search Article: issta26main-p1604-p (type: Full Paper) doi:10.1145/3832240

Gleaner: A Semantically-Rich and Efficient Online Sampler for Microservice Diagnostics
Yifan Yang, Aoyang Fang, Songhan Zhang, and Pinjia He
(Chinese University of Hong Kong, Shenzhen, China)
Distributed tracing in microservices is critical for diagnostics but generates overwhelming data volumes, necessitating intelligent sampling. To maximize fidelity, state-of-the-art (SOTA) tail-based samplers analyze complete (or even log-enriched) traces by modeling them as graphs. However, this reliance on computationally expensive graph analysis creates a performance bottleneck that prohibits their use in online settings.
To this end, we propose Gleaner, an online tail-sampling framework that breaks this trade-off. It is founded on the key insight that explicit graph structures are unnecessary for high-fidelity trace grouping. Instead, Gleaner represents each trace as a “bag-of-edges” augmented with log semantics, replacing slow graph algorithms with highly efficient set-based operations. It also employs an alarm-driven quota and a diversity-preserving strategy to prioritize anomalous and rare traces for downstream Root Cause Analysis (RCA). Experimentally, Gleaner processes traces at 0.74ms each, improving Trace Pattern Coverage by up to 128.7% and Shannon Entropy by up to 32.9% over baselines. At just a 1% sampling rate, Gleaner improves RCA accuracy by 42%-107% over the next-best sampler. Moreover, RCA on Gleaner’s sampled data is more accurate than with the entire, unsampled dataset. This result reframes intelligent sampling from a data reduction technique to a powerful signal enhancement paradigm for automated operations.
Article Search Artifacts Available Article: issta26main-p1619-p (type: Full Paper) doi:10.1145/3832241

Artifact for "Gleaner: A Semantically-Rich and Efficient Online Sampler for Microservice Diagnostics" (doi:10.5281/zenodo.21063794): Gleaner is a semantically rich online sampler for microservice diagnostics. This artifact supports the ISSTA 2026 Artifact Evaluation by packaging the Gleaner implementation, Platform integration, reduced live inputs, full-dataset inputs, third-party baseline/RCA code, and scripts that regenerate the artifact evidence ...

Context Matters: Improving the Practical Reliability of LLM-Based Unit Test Generation (Experience Paper)
Junjie Chen, Ziqi Wang, Lin Yang, Chen Yang, Xiao Chu, Jianyi Zhou, Guangtai Liang, Qianxiang Wang, and Dong Wang
(Tianjin University, China; Huawei Cloud, China)
Automated unit test generation has recently benefited from advances in large language models (LLMs), yet our industrial deployments reveal a persistent gap between promising research results and practical usability. In real-world projects with complex frameworks and cross-file dependencies, LLM-generated tests frequently fail to compile, require costly manual repair, or provide unstable coverage improvements. This paper reports our experience in designing, deploying, and evaluating CATGen, a context-aware workflow for LLM-based unit test generation, informed by repeated industrial failures and refinements. Rather than relying on LLMs to infer incomplete project context, we found that compilation robustness critically depends on making project-level dependencies explicit, stabilizing test class scaffolding, and replacing iterative LLM-based repair with lightweight static analysis. These experience-driven insights shaped CATGen’s multi-stage design, which combines structured context retrieval, deterministic test skeleton construction, and program analysis–based post-processing. We evaluate CATGen on real-world complex focal methods from proprietary industrial projects and additionally on the Defects4J benchmark to assess generalizability. Across both settings, CATGen substantially improves compilation success and structural coverage while significantly reducing generation time and token consumption compared to existing LLM-based approaches. Our results demonstrate that reliable LLM-based unit test generation in practice depends less on prompt engineering alone and more on systematic engineering support grounded in real-world development constraints.
Article Search Article: issta26main-p1633-p (type: Full Paper) doi:10.1145/3832242

Inferring 1-Minimal Trigger Configurations for Assessing Linux Kernel CVE Triggerability
Tongjie Wei, Peng Zhang, Zhiwen Hu, Xupu Hu, Chen Lyu, and Gangyan Zeng
(Nanjing University of Science and Technology, China; Shandong Normal University, China)
Vendors assessing Linux kernel CVEs need to know whether a bug is triggerable under production-tailored configurations, not merely whether a version is affected, yet upstream reproducers and vulnerability databases rarely provide configuration-level context. We study minimal trigger-configuration inference: given a CVE entry and a target kernel version (optionally a baseline .config), we synthesize a Kconfig-satisfiable option set that remains effective after make olddefconfig and, when a reproducer is available, still triggers under a specified evaluation protocol; we then prune it to a 1-minimal (subset-minimal) boundary for evaluation. Our framework FCC links vulnerability cues to build-system symbols, completes implicit prerequisites under olddefconfig feedback to avoid silent rollback, and performs runtime-validated minimization guided by dependency topology. We evaluate on KernJC and KernelCTF, totaling 88 CVEs across multiple kernel versions. On the 88-CVE set, FCC improves the post-make olddefconfig configuration success rate from 62.5% (55/88) to 96.6% (85/88) over an olddef-only injection baseline; on the KernJC set, FCC reduces the average candidate set size by 78.7% compared to KernJC (Avg. 14.72 vs. 69.00 options per CVE). A stage-wise analysis of time and token costs shows that Stage I dominates overhead, while CVE-focused evidence selection substantially reduces this cost. By returning an effective and auditable 1-minimal configuration boundary, FCC helps vendors scope triggerability against their deployment configurations with a clear, tool-supported decision line.
Article Search Artifacts Available Article: issta26main-p1636-p (type: Full Paper) doi:10.1145/3832243

Reproduction Package for Article "Inferring 1-Minimal Trigger Configurations for Assessing Linux Kernel CVE Triggerability" (doi:10.5281/zenodo.20988950): This is the reproduction artifact for the ISSTA 2026 paper "Inferring 1-Minimal Trigger Configurations for Assessing Linux Kernel CVE Triggerability". It implements FCC, a three-stage large language model driven framework that automatically infers 1-minimal kernel configuration sets for stably triggering Linux kernel ...

The Discreet Charm of the Bugeoisie: A First Look at Bug Reports Created by Researchers
Ji young Kim, Jana Dragovic, Alessandro Botta, T. M. Rithwanul Islam, Alaa Mohamad, Karim Sharaf, Sejuti Sharmin Siddiqui, Divyanshi Joshi, Harini Anand, Nurjemal Saryyeva, Shubham Chapagain, Saad Nasir, Darko Marinov, and Bogdan Alexandru Stoica
(University of Illinois at Urbana-Champaign, USA; University of Texas at Dallas, USA; Jahangirnagar University, Bangladesh; American University of Beirut, Lebanon; Egyptian E-Learning University, Egypt; Alexandria University, Egypt; University of Dhaka, Bangladesh; Maharaja Agrasen Institute of Technology, India; PES University, India; National University of Singapore, Singapore; Tribhuvan University, Nepal; American International University-Bangladesh, Bangladesh)
Many ISSTA papers present automated tools to find software bugs. Some researchers create bug reports for the bugs they deem worth reporting, e.g., new bugs found by the tools. We study bug reports created by researchers and coin the term Bugeoisie to refer to such bug reports. We collect the Bugeoisie for ISSTA 2025 by reading the papers, examining the artifacts, and contacting the authors, trying to “chase” bug report links. This process turns out to be surprisingly challenging because only a few authors make such links readily available. We then analyze the Bugeoisie to understand potential bad practices. For most papers that claim some bugs reported or confirmed, independently checking the claims is again surprisingly challenging. We also start analyzing Bugeoisie for FSE 2025 to check generalizability of our findings. Our findings point out to the important changes that conference organizers and reviewers may want to require for the Bugeoisie. We also provide some suggestions for paper authors to prepare and describe their Bugeoisie in their papers and artifacts.
Article Search Article: issta26main-p1662-p (type: Full Paper) doi:10.1145/3832244

BinRAG: An RAG-Based Decompilation Framework Fusing Name Prediction and Calling Context
Wai Kin Wong, Daoyuan Wu, Zhibo Liu, Huaijin Wang, Zongjie Li, and Shuai Wang
(Hong Kong University of Science and Technology, China; Lingnan University, Hong Kong; Nanjing University, China; Shandong University, China)
Decompiling stripped binaries to assist human reverse engineers remains a critical yet highly challenging task in software security. Prior work has developed various neural networks and even dedicated large language models (LLMs) to improve function and variable name recovery in decompiled code. Nonetheless, these approaches alone fall short of enabling a generic LLM-based decompilation pipeline that can reliably enhance the readability and semantic clarity of decompiled outputs.
In this paper, we propose BinRAG, a retrieval-augmented generation (RAG) based decompilation framework designed to enhance the decompilation of stripped binaries. Building upon name prediction models, BinRAG features three novel designs: (1) it first utilizes name prediction models to transform raw decompiled outputs into enriched, source-like queries for RAG retrieval; (2) it further enhances these queries using a fine-tuned specialized LLM conditioned on the predicted variable names, enabling more accurate retrieval from a curated example database; (3) it then integrates these retrieved examples with the target’s calling context, employing a general-purpose LLM to synthesize high-fidelity, human-readable decompiled code. Evaluation on 3,200 functions from real-world software repositories demonstrates that BinRAG improves readability by 8.4% over standard RAG and semantic precision by 31.2% over the next-best prior baseline. Our results show that BinRAG effectively scales to large codebases and significantly reduces manual effort in reverse engineering tasks.
Article Search Article: issta26main-p1665-p (type: Full Paper) doi:10.1145/3832245

Exploiting Ethereum Rollback Semantics: Profit-Driven Attack Synthesis and Off-Chain Misinterpretation Testing
Yixuan Liu, Xinlei Li, and Yi Li
(Nanyang Technological University, Singapore)
The Ethereum Virtual Machine (EVM) enforces atomic execution through rollback, reverting all state changes when execution fails. While necessary for correctness, rollback semantics introduce a distinct attack surface affecting both on-chain execution and off-chain infrastructures. On-chain, attackers can use conditional failure to filter executions, committing only profitable outcomes while rolling back unprofitable attempts. Off-chain, systems such as explorers, token trackers, and RPC providers may misinterpret aborted executions as successful, leading to inconsistent records or unintended transfers. Existing tools largely treat rollback as an execution endpoint, providing limited support for profit-driven attack synthesis or off-chain misinterpretation testing. To address this gap, we formalize two rollback attack models and develop RollGain, a unified framework for rollback-aware analysis. For on-chain attacks, RollGain models contract structure and value flows, validates candidate executions symbolically, and replays on a forked chain to rank profitability. For off-chain testing, RollGain conducts call tree analysis on 3.08 billion Ethereum transactions to characterize rollback patterns and exercises rollback-inducing execution vectors against external services. On our evaluation datasets, RollGain achieves 95.3% recall with zero false positives, and uncovers 20 rollback misinterpretation vulnerabilities across 18 off-chain systems, of which 18 have been confirmed, 16 fixed, and 5 assigned CVE identifiers.
Article Search Article: issta26main-p1691-p (type: Full Paper) doi:10.1145/3832246

Enhancing LLM-Based Bug Reproduction via Code Entity Retrieval and Test Case Repair
Hao Ding, Yanjie Jiang, Yuxia Zhang, and Hui Liu
(Beijing Institute of Technology, China; Tianjin University, China)
Automated bug reproduction from bug reports is a critical yet challenging step in software debugging. While LLM-based bug reproduction shows promise, its effectiveness is often hampered by insufficient contextual awareness of the relevant codebase and a tendency to produce invalid test cases. To address these limitations, we propose a novel approach, called LTER, that enhances LLM-based bug reproduction through fine-grained code entity retrieval and a feedback-driven dynamic repair loop. LTER first identifies specific code entities within bug reports to automatically extract precise contexts, including class definitions, constructors, and method logic. The extracted contexts are then used to guide the LLM in generating reproduced test cases. To further ensure executability, LTER employs an iterative repair mechanism to resolve complex dependencies. Specifically, upon injecting a generated test case into the project, if a compilation failure occurs, the framework forwards the error messages to the LLM for an initial repair. Should this initial repair fail, it empowers the LLM to analyze diagnostic messages to recognize missing context and retrieve indispensable dependencies, subsequently regenerating the test case with the supplemented data. Finally, LTER employs a hybrid cascade ranking strategy to accurately select the most effective reproduction test case from the generated candidates. The experimental results on the widely-used Defects4J benchmark show that LTER substantially outperforms the best performance in automated bug reproduction, increasing the reproduction success rate to 46.2% with successfully identifying a valid reproduction test as the top candidate in 38.1% of the cases. Furthermore, LTER demonstrates strong generalization capability, delivering robust performance on the GHRB dataset containing recent bugs previously unseen by the LLM.
Article Search Article: issta26main-p1713-p (type: Full Paper) doi:10.1145/3832247

DocPrism: Multi-lingual Detection of Incorrectness Inconsistencies between Code and Documentation
Xiaomeng Xu, Zahin Wahab, Reid Holmes, and Caroline Lemieux
(University of British Columbia, Canada)
Code-documentation inconsistencies are common and undesirable: they can lead to developer misunderstandings and software defects. This paper introduces DocPrism, a lightweight multi-language, code-documentation inconsistency detection tool. DocPrism uses a standard large language model (LLM) to analyze and explain inconsistencies, and focuses on outputting incorrectness inconsistencies. Plain use of LLMs for this task yields unacceptably high inconsistency flag rates—i.e., over 90% of functions are flagged as inconsistent with their documentation. One substantial reason is that LLMs identify natural gaps between high-level documentation and code as incompleteness inconsistencies. We introduce and apply the Local Categorization, External Filtering (LCEF) methodology: LCEF uses an LLM’s local completion skills, rather than its long-term reasoning skills, to focus on reporting incorrectness inconsistencies. In our ablation study, LCEF reduces DocPrism’s inconsistency flag rate from 98% to 14%, and increases F1 score from 0.22 to 0.77, compared to standard prompting techniques. On a broad evaluation across Python, TypeScript, C++, and Java, DocPrism maintains a low flag rate of 17%, and achieves a precision of 0.63 without performing any fine-tuning. We also establish a conservative lower bound across four programming languages, showing that inconsistency errors are present in 11% of code-documentation pairs. In addition, DocPrism achieves precision comparable to the state-of-the-art on an established synthetic dataset, but substantially outperforms it on our real-world Java dataset in precision (DocPrism: 0.47–0.67 vs. SOTA: 0.05–0.14).
Article Search Article: issta26main-p1715-p (type: Full Paper) doi:10.1145/3832248

NCFuzz: Configuration-Guided Network Service Fuzzing
Xuesong Bai, Hengkai Ye, Shenghan Zheng, Fenglu Zhang, Hong Hu, and Zhou Li
(University of California at Irvine, USA; Pennsylvania State University, USA; Dartmouth College, USA; China Telecom, China)
Network services like FTP and DNS are critical components of modern reliable Internet infrastructure. Software fuzzing, especially network protocol fuzzing, is widely used to uncover flaws in these systems. However, conventional fuzzers operate under a single, fixed configuration throughout the fuzzing campaign, leaving the service’s rich configuration space unexplored. Incorporating configurations as a dynamic input dimension is challenging due to complex semantics, trigger conditions, and the resulting enlarged search space. We tackle the problem of finding bugs under non-default configurations, termed ConfBug, by designing a new fuzzer called NCFuzz. The non-default configurations can be uncommon but administrators may enable them, which cannot be exercised by conventional fuzzers. With the assumption that software documentation that describes configuration options is available, NCFuzz leverages two key observations: 1) software documentation contains rich information about configurations; 2) interactions between configuration and network messages can be tracked through code instrumentation and data-flow analysis. Using these insights, NCFuzz uses configuration knowledge and the relationships between configurations and network messages to guide the fuzzer toward new software states. The quality and completeness of the documentation will affect the effectiveness of NCFuzz. Evaluation on six network service implementations shows NCFuzz achieves higher coverage than baseline fuzzers. Five ConfBugs were discovered during fuzzing.
Article Search Article: issta26main-p1733-p (type: Full Paper) doi:10.1145/3832249

Re-evaluating Detection of Equivalent Mutants using LLMs: We Should Properly Measure How Far We Are
Arjun Tandon, Mehmet Fırat Dündar, Milkiyas Gebremichael Gebru, Darko Marinov, Yiling Lou, and Wenxi Wang
(Indraprastha Institute of Information Technology Delhi, India; Sabancı University, Türkiye; Addis Ababa University, Ethiopia; University of Illinois at Urbana-Champaign, USA; University of Virginia, USA)
Mutation testing is a widely used approach for measuring test-suite quality. A critical problem in mutation testing is equivalent mutant detection (EMD), i.e., determining if a mutant semantically behaves the same as the original code despite some syntactic differences. A recent study has shown that LLM-based EMD techniques hold great promise, reporting substantial improvements over traditional compiler- and machine-learning–based approaches. In this work, we revisit those recent results and evaluate the generalization capabilities of the proposed LLM-based EMD techniques across two additional datasets that differ from the prior dataset in mutation operators, programming languages, or source projects. Contrary to prior findings, the proposed LLM-based EMD techniques suffer substantial performance degradation on the two additional datasets. Through an extensive analysis, we identify a key factor underlying the differences as original-method–level data leakage (i.e., the same original method appearing in both training and testing sets), indicating that prior results under within-method evaluation do not generalize to cross-method evaluation. We find that the studied LLMs tend to rely on a method-wise majority-voting shortcut rather than reasoning about the semantic effects of mutations. Based on these findings, we call for the adoption of realistic cross-method evaluation and the development of mutation-centric semantic reasoning in future LLM-based EMD research.
Article Search Artifacts Available Article: issta26main-p1765-p (type: Full Paper) doi:10.1145/3832250

Reevaluating-Detection-of-Equivalent-Mutants-Using-LLM (doi:10.6084/m9.figshare.32835686.v3): It presents the repository structure, maps paper claims to supporting artifacts, and provides step-by-step instructions to reproduce all results.

CID: Clean-Seed-Free Backdoor Defense via Counterfactual Invariance for Neural Code Models
Junyao Ye, Zhen Li, Xi Tang, Shulin Li, Shi Liang, Deqing Zou, and Hai Jin
(Huazhong University of Science and Technology, China)
Neural code models automate core software engineering tasks such as code classification and generation, but remain vulnerable to backdoor attacks. Existing defenses struggle with both injection-based and Semantically-Equivalent Transformation (SET)-based triggers and often require a trusted, pre-verified in-distribution clean seed set, which can be costly to obtain in third-party fine-tuning. This paper introduces Counterfactual Invariance-based Defense (CID), a clean-seed-free defense requiring no a priori trusted in-distribution clean data and grounded in operational counterfactual invariance tests. CID exploits asymmetric counterfactual behavior: clean predictions tend to degrade under semantic context corruption, whereas backdoor predictions show larger representation drift when shortcut-carrying structures are neutralized. Accordingly, CID applies two orthogonal probes: a contextual intervention for semantic-sufficiency testing and a gradient-guided structural intervention for representation-drift testing, to extract a high-purity clean seed set directly from a mixed dataset. CID then uses these seeds to calibrate representation-space filtering over the full dataset. Across four software engineering tasks, six trigger instantiations, poisoning rates from 1% to 10%, multiple model architectures, and multilingual code summarization, CID achieves high poison-detection performance relative to evaluated baselines while keeping false positives low in most settings. Clean-only and selected retraining experiments further show conservative benign-data retention, preserved clean-task utility, and reduced residual attack success rate in challenging Defect Detection settings.
Article Search Article: issta26main-p1771-p (type: Full Paper) doi:10.1145/3832251

Provably Lossless Acceleration of DNN Mutation Testing via Memoization
Ali Ghanbari, Ben Greenman, Sasan Tavakkol, and Shibbir Ahmed
(Auburn University, USA; University of Utah, USA; Google Research, USA; Texas State University, USA)
Mutation analysis has recently reemerged in the context of deep neural networks (DNNs) as a promising, but notoriously costly, approach for assessing test dataset adequacy. Existing techniques speed up DNN mutation testing through lossy approximations that trade efficiency for mutation score accuracy. This paper introduces Mure, the first provably lossless framework for accelerating DNN mutation testing via memoization. Mure is based on the idea that DNN mutants and the original model share substantial redundant computation, so during mutation testing, it executes only the mutated suffixes of each mutant and reuses the common prefix from the original model, which is computed only once. We give a formal account of memoized mutation testing, and prove that Mure is sound, i.e., it produces results equivalent to exhaustive vanilla mutation testing, and identify basic conditions under which speed-up is guaranteed. We have implemented Mure and evaluated it on 15 DNN models of various architectures, complexities, and sizes ranging from a few thousands to millions of parameters. This provides empirical evidence that Mure reduces the computational cost of mutation testing by 44.54%, on average. We also observed that while state-of-the-art techniques tend to yield higher acceleration (up to 88.97%, on average), they come at the cost of some error in mutation score. We further analyze the effect of mutation generation selection ratio on the effectiveness of Mure and observed predictable reductions in memoization opportunities with increasing the percentage of mutated neurons. We observed that Mure offers more than 20% speed-up even when as high as 5% of the neurons are mutated.
Article Search Article: issta26main-p1785-p (type: Full Paper) doi:10.1145/3832252

Profiling-Guided Bayesian Optimization of JVM Configurations
Abdelrahman Baz, Wing Lam, and August Shi
(University of Texas at Austin, USA; George Mason University, USA)
Regression testing is essential for maintaining software quality but often incurs substantial time costs. While regression testing time can often be reduced by selectively running fewer tests, prior work has demonstrated that tuning Java Virtual Machine (JVM) configuration flags can also reduce testing time in Java projects, even while running all tests and preserving original testing outcomes. However, finding effective flag combinations remains challenging due to the vast configuration space and complex interactions between flags. Random search and direct modeling approaches that map flag configurations to testing time have shown limited effectiveness in navigating this complex optimization landscape.
We present PROBO (PROfiling-Guided Bayesian Optimization), an iterative approach that leverages JVM runtime metrics (e.g., garbage collection frequency, just-in-time (JIT) compilation rates, and memory allocation) to guide Bayesian optimization for testing time reduction. Unlike prior work using Bayesian inference that directly models the relationship from flag configurations to testing time, PROBO decomposes the prediction problem through observable runtime behaviors: a metrics model predicts how flag configurations affect runtime metrics, and a performance model predicts how those metrics affect testing time. PROBO collects 44 runtime metrics using a profiler during test execution, and propagates feature importance scores through both models to identify which flags most strongly influence testing time-predictive metrics. Finally, PROBO generates candidate flag configurations through three complementary strategies guided by expected testing time improvement.
We evaluate PROBO on 16 open-source Java projects, comparing against random search and BOCA (a Bayesian optimization baseline). PROBO achieves an average testing time reduction of 10.7% across all projects when evaluating 20 configurations per project, outperforming random search (5.8%) by 1.85× and BOCA (4.3%) by 2.49×. PROBO successfully generates configurations that substantially reduce testing time for all 16 projects, with reductions ranging up to 27.4%. With a one-hour time budget for search, PROBO maintains its advantage with 8.0% average reduction, demonstrating practical applicability. PROBO-generated configurations remain effective across software evolution, maintaining 8.8% average reduction over an average of 88 future commits per project. Our analysis reveals that metrics related to JIT compilation, particularly Total Compilation Rate (methods compiled per second) and C1 Compilation Rate (first-tier JIT compilation rate), are the strongest predictors of testing time, accounting for 67.2% of consistently important metrics across projects.
Article Search Article: issta26main-p1799-p (type: Full Paper) doi:10.1145/3832253

Understanding and Improving Model Editing for Secure Code Generation
Weifeng Sun, Quanjun Zhang, Yuchen Chen, Chengran Yang, Gou Tan, and David Lo
(Singapore Management University, Singapore; Nanjing University of Science and Technology, China; Nanjing University, China; Sun Yat-sen University, China)
Large language models (LLMs) are widely used for code generation, yet they can reproduce vulnerable code implementations learned from insecure patterns in training data. Prior work has primarily explored inference-time hardening to reduce insecure generations without updating the target model. While effective, this paradigm couples security behavior to the auxiliary component and incurs additional runtime overhead. This paper presents the first systematic empirical study of applying model editing as the model-level hardening mechanism for secure code generation. Unlike inference-time interventions, model editing updates a small subset of parameters to inject security-relevant knowledge directly into the target LLM. We evaluate 3 state-of-the-art editing methods across diverse LLM families and compare them with CoSec, a representative inference-time hardening approach, focusing on: (i) security effectiveness and robustness, (ii) generalization to unseen vulnerabilities, and (iii) functional correctness on general programming tasks. Our results show that model editing yields substantially larger security gains than CoSec on seen vulnerability types, improving security ratios by 15
Article Search Article: issta26main-p1843-p (type: Full Paper) doi:10.1145/3832254

Multi-stage On-Demand Program Slicing for Modular Analysis of Multi-threaded Programs
Jiawei Yang, Xiao Cheng, Jiawei Wang, Xiapu Luo, and Yulei Sui
(UNSW, Australia; Macquarie University, Australia; Hong Kong Polytechnic University, China)
Precise analysis of multi-threaded programs requires combining flow-sensitive pointer analysis (FSPTA) with interleaving and lock analysis (ILA) to reason about cross-thread value flows under feasible concurrent executions. ILA computes may-happen-in-parallel (MHP) relations and lock-release spans to determine when shared accesses can occur concurrently. Unfortunately, these analyses are both expensive and tightly coupled: FSPTA needs ILA to rule out infeasible inter-thread def-use relations, while ILA needs alias information to identify interference-relevant interactions. As a result, whole-program analyses often spend most of their time on code that is irrelevant to the client query. We present MSli, an on-demand slicing framework for modular analysis of multi-threaded programs. It extracts compact, query-relevant program slices while preserving the answers of downstream analyses. Unlike single-pass slicing over a unified dependence graph, MSliperforms multi-stage slicing with analysis-specific criteria. Concretely, a lightweight pre-analysis establishes an over-approximation of inter-thread value flows and performs ILA slicing source extraction to identify the MHP and lock-span queries required later for ILA slicing. The refined main-phase ILA results then enable reconstruction of a thread-aware value-flow graph to guide FSPTA slicing, supporting modular analysis and downstream clients. We implement MSliin SVF and evaluate it on ten large real-world projects with data race detection as a representative client. Compared with the unsliced baseline (FSAM), MSlireduces the analyzed ICFG to 5.4% (ILA) and 25.7% (FSPTA), reduces ILA/FSPTA runtimes to 4.7%/18.3%, and cuts total analysis time to 20.8% on average, while producing identical query outcomes and race alarms.
Article Search Article: issta26main-p1853-p (type: Full Paper) doi:10.1145/3832255

ProgSCA: Software Composition Analysis via Program-Level Modeling
Peihong Li, Cheng Li, Yuchen Gu, Yanzhe Hu, Liheng Chen, Zeyu Gao, Hao Wang, and Chao Zhang
(Tsinghua University, China; Huazhong University of Science and Technology, China; Beijing University of Posts and Telecommunications, China; Institute of Information Engineering at Chinese Academy of Sciences, China; Peking University, China; JCSS - Tsinghua University (INSC) - Science City (Guangzhou) Digital Technology Group Co. Ltd., China)
Software composition analysis (SCA) aims to identify third-party dependencies in programs, which plays a critical role in ensuring software supply chain security. Existing approaches largely follow a rule-based paradigm: they first compute function-level similarities, then aggregate these results using handcrafted heuristics to determine which third-party libraries (TPLs) the target program depends on. However, such rules require substantial manual effort and expert knowledge to design, tune, and maintain. To address this, we present ProgSCA, an SCA framework based on a two-stage strategy and program-level rather than function-level modeling. ProgSCA formulates SCA as a retrieval problem, first employing lightweight methods to quickly filter out irrelevant libraries from numerous candidate TPLs, then using a model trained at the program level to directly predict dependency between programs. Comprehensive evaluations show that ProgSCA achieves state-of-the-art performance in SCA tasks, improving F1 scores over existing methods by 174% and 100% in two mainstream scenarios, respectively. Moreover, ProgSCA maintains a consistent advantage across different datasets and different candidate pool scales, and also proves effective in the downstream task of function similarity matching, further demonstrating the practical value of our approach.
Article Search Article: issta26main-p1863-p (type: Full Paper) doi:10.1145/3832256

SmartDecompiler-R1: Enhancing Faithful and Explainable Smart Contract Bytecode Decompilation with Reinforcement Learning
Yilun Ma, Lingxiao Tang, Li Lin, Zhipeng Gao, Jiachi Chen, Xin Xia, and Lingfeng Bao
(Zhejiang University, China; Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security, China)
Understanding EVM bytecode is critical for smart contract security analysis. Existing decompilers typically rely on heuristic rules or leverage large language models (LLMs) to generate source code after bytecode analysis. However, heuristic-based approaches often produce pseudocode that is difficult for humans to interpret, while LLM-based methods also face several problems. LLMs have little experience with low-level bytecode, which weakens their reasoning and causes inaccurate results. Additionally, their tendency to auto-correct code breaks faithfulness to the original program. A general lack of clarity in these approaches also hinders effective auditing and interpretation. In this paper, we propose SmartDecompiler-R1, an end-to-end decompilation framework that translates Three-Address Code (TAC), a register-based representation of EVM bytecode, into source code using reinforcement learning. SmartDecompiler-R1 significantly improves both the accuracy and consistency of decompiled code, while additionally providing human-readable explanations for the bytecode-to-source generation process. To the best of our knowledge, we are the first to design a benchmark equipped with well-defined test cases and a systematic evaluation framework for smart contract decompilation. Experimental results on this benchmark demonstrate that SmartDecompiler-R1 substantially outperforms existing decompilers on execution consistency by 46.23%. In particular, compared with LLM-based approaches, SmartDecompiler-R1 achieves a significant improvement in preserving vulnerability consistency between the original and decompiled contracts. Furthermore, we conduct extensive ablation studies to validate the effectiveness of individual components of SmartDecompiler-R1, and design targeted experiments to demonstrate the interpretability and readability of the generated explanations.
Article Search Article: issta26main-p1868-p (type: Full Paper) doi:10.1145/3832257

Revisiting Graph Representations for ML-Based Binary Code Similarity Detection: A Systematic Study
Tengteng Yang, Yikun Hu, Jican Zhang, Lei Xue, Ming Fan, and Liang Zhang
(Sun Yat-sen University, China; Shanghai Jiao Tong University, China; Shenzhen Loop Area Institute, Shenzhen, China; Xi'an Jiaotong University, China)
Binary Code Similarity Detection (BCSD) is a foundational capability in software security, underpinning critical applications ranging from vulnerability detection to malware analysis. While recent tools based on Machine Learning (ML) have achieved significant performance improvements, their efficacy is heavily contingent upon the underlying code representation. Through a systematic literature review of ML-based BCSD papers, we find that existing approaches typically leverage linear sequences or adopt graph-based representations, with the latter constituting the majority (77%). Despite this prevalence, there is no consensus on which graph representation yields superior effectiveness. Existing works usually couple graph construction with customized learning backbones and evaluate them on inconsistent benchmarks. This makes isolating the representation's impact difficult. Consequently, determining which graph topologies most effectively capture robust binary-code semantics under controlled and comparable evaluation settings remains an open problem.
In this paper, we present a systematic study of graph representations for ML-based BCSD to bridge this gap. Specifically, we implement a modular evaluation framework that decouples graph construction from model training. Using this framework, we systematically evaluate seven representative graph representations, finding that no single representation is universally dominant, that distinct topologies exhibit unique strengths depending on the evaluation scenario, and that their rankings are largely backbone-stable despite varying absolute performance. We further investigate their combination effectiveness in N-day vulnerability detection and employ a tailored post-hoc analysis tool to study model-level structural reliance. The results show that DFG, PDG, and SOG subgraph pairs more often preserve trained models' similarity scores under pruning, while several other representations are more sensitive to structural reduction.
Article Search Article: issta26main-p1876-p (type: Full Paper) doi:10.1145/3832258

SEDCoT: Enhancing LLM-Based COBOL Code Translation via Symbolic Execution and Delta Debugging
Phillip Entin, Wenchao Gu, Alexander Knapp, and Chunyang Chen
(USI Lugano, Switzerland; TU Munich, Germany; University of Augsburg, Germany)
COBOL remains critical across banking, insurance, and government infrastructure. However, maintenance is increasingly challenging due to outdated technologies, sparse documentation, and developer retirement, necessitating code translation into modern languages like C. Traditional rule-based transcompilers yield outputs that are difficult to read and maintain, while general-purpose large language models (LLMs) achieve suboptimal correctness because COBOL is a low-resource language with distinct logic patterns. To bridge this gap, we propose SEDCoT, a novel COBOL-to-C translation framework. SEDCoT first leverages LLMs for initial translation, then combines symbolic execution with LLM guidance to generate test suites and iteratively repair semantic discrepancies. Finally, it integrates delta debugging to minimize failing tests into succinct counterexamples, accelerating automated code repair. Evaluating SEDCoT on a public COBOL-to-C dataset demonstrates that it outperforms state-of-the-art baselines by at least 12% while producing translations with substantially higher readability than rule-based alternatives.
Article Search Article: issta26main-p1883-p (type: Full Paper) doi:10.1145/3832259

ConUT: Condition-Aware Test Generation for Complex Java Code
Ruiguo Yu, Ruiqi Dong, Xi Xiao, Xiaogang Zhu, Shaohua Wang, Sheng Wen, and Qing Li
(Tsinghua University, Shenzhen, China; Peng Cheng Laboratory, Shenzhen, China; Swinburne University of Technology, Australia; Adelaide University, Australia; Central University of Finance and Economics, China)
Abstract: Automatically generating high-coverage unit tests for complex Java methods remains a formidable challenge, particularly when execution paths are guarded by intricate control-flow nesting and cross-class state dependencies. Existing LLM-based approaches predominantly follow a goal-driven paradigm, relying on unguided code synthesis within a vast search space to satisfy coverage objectives. Lacking explicit guidance on how to construct the precise object states required for complex predicates, these methods suffer from a construction-reachability gap, ultimately resulting in insufficient coverage for complex methods. This paper introduces ConUT, a condition-aware unit test generation framework centered on the paradigm of Configuration-Driven Guidance, which is a strategy that bridges the gap between abstract branch targets and concrete code realization by providing the LLM with structured instantiation recipes. Our core insight is that the implicit causal chain leading to a target branch can be explicitly manifested via static analysis and distilled into structured guidance. ConUT utilizes Backward Dependency Tracing (BDT) to identify the fields and method calls that influence predicates, and distills these findings into structured Configuration Templates (C_cfg). These templates serve as a “semantic roadmap,” pruning the LLM’s search space and transitioning it from a stochastic generator into a guided logic realizer.
We evaluate ConUT on eight real-world Java projects characterized by high-complexity “logic hotspots.” Experimental results demonstrate that ConUT outperforms state-of-the-art baselines, achieving an average branch coverage of 72.84% and line coverage of 83.87%. Notably, ConUT exceeds the best baseline in average branch coverage by 11.94 percentage points while utilizing 67% fewer test cases.
Article Search Article: issta26main-p1921-p (type: Full Paper) doi:10.1145/3832260

Testing Method Relocation Algorithms via Template-Based Systematic Structural Traversal and Precondition Filtering
Chunhao Dong, Yanjie Jiang, Yang Zhang, and Hui Liu
(Beijing Institute of Technology, China; Tianjin University, China; Hebei University of Science and Technology, China)
Method relocation refactorings, primarily Move Method and Pull Up/Push Down Method, are indispensable for reducing coupling and enhancing cohesion. Despite their widespread automation in modern refactoring engines, these algorithms remain notoriously error-prone, posing significant risks to software reliability. A primary challenge in testing them lies in the vast search space of complex program structures and the intricate preconditions required for safe method relocation. To address this, we propose RelocTest, a comprehensive testing framework that combines template-driven structural traversal with automated precondition filtering. RelocTest systematically explores the input space by populating program templates specially designed for method relocation through a two-stage generation process: (1) Skeleton Synthesis, which systematically traverses diverse syntactic structures, and (2) LLM-Guided Completion, which leverages Large Language Models to inject diverse, executable code into these skeletons. This hybrid strategy ensures high structural coverage while maintaining test program validity. Furthermore, to optimize testing efficiency, we introduce an LLM-based Precondition Extractor that analyzes the implementation of method relocation algorithms to identify and prune test programs destined for rejection. We evaluated RelocTest on 7 mainstream refactoring engines. Our approach successfully uncovered 56 previously unknown bugs, with 19 already confirmed by tool vendors, demonstrating its effectiveness in hardening industrial-strength refactoring tools.
Article Search Article: issta26main-p1939-p (type: Full Paper) doi:10.1145/3832261

E2WR: An Effective and Efficient Reduction Framework for WebAssembly Binaries
Shiyao Zhou, Ningyu He, David Lo, and Xiapu Luo
(Hong Kong Polytechnic University, Hong Kong; Singapore Management University, Singapore)
WebAssembly (Wasm) is a prominent programming language enabling high-performance execution across diverse computing environments. However, bugs in Wasm runtimes, which execute Wasm binaries, can lead to severe security breaches and system failures. Manually debugging Wasm binaries that trigger such runtime bugs is exceedingly difficult due to their poor human readability, stemming from low-level stack-based instructions, complex control flow structures, and extraordinary length. Thus, there is a critical need for automated reduction techniques that minimize Wasm binaries while preserving the ability to trigger the original bug. However, existing reducers often suffer from significant limitations in effectiveness and efficiency: language-agnostic reducers frequently generate invalid variants without Wasm validation awareness, while Wasm-specific reducers still lack efficient intra-function instruction-sequence reduction and reduce definitions inefficiently. To address these challenges, we propose E²WR, an effective and efficient reduction framework for Wasm binaries. E²WR introduces a two-stage approach for intra-function instruction-sequence reduction, consisting of operand-dependency guided instruction reduction and combination-aware redundancy elimination to enable efficient delta debugging while avoiding unnecessary padding instructions, and a definition reduction approach that combines static analysis with delta-debugging-guided trials to remove property-irrelevant definitions efficiently. Compared with Wasm-Shrink and Wasm-Reduce, E²WR produces binaries that are 95.0% and 68.6% smaller, respectively, and achieves reduction speedups of 7.9×–12.4×.
Article Search Article: issta26main-p1945-p (type: Full Paper) doi:10.1145/3832262

LogNexus: Effective Log Compression via Unified Redundancy Encoding
Yang Liu, Kaiming Zhang, Zhuangbin Chen, and Zibin Zheng
(Sun Yat-sen University, China)
State-of-the-art log compressors typically rely on a decoupled “parse-then-compress” workflow, where parsing is optimized for semantic accuracy (i.e., event identification) rather than storage efficiency. Through a comprehensive empirical study, we reveal that this architectural decoupling prevents the exploitation of deep correlations between static templates and dynamic variables. To address it, we propose LogNexus based on the principle of unified redundancy encoding, a new log compression paradigm that co-designs structural extraction and variable encoding. LogNexus constructs a Unified Redundancy Tree (URT) using a hierarchical strategy that progressively mines frequent “structure+variable” patterns in logs. Such a design captures deep contextual redundancies ignored by traditional methods while minimizing computational overhead by pre-emptively encoding dominant patterns. Extensive evaluation on 16 benchmark datasets demonstrates that LogNexus establishes a new state-of-the-art. It achieves the highest compression ratio on 14 datasets (outperforming baselines by 9.48%–89.13%) and the fastest speed (1.51×–40.06× faster than competitors). Furthermore, when configured in non-chunked mode to maximize global pattern discovery, LogNexus boosts its compression ratio by 285.13%, which is 27.08% higher than the best baseline, while retaining a 2.43× speed advantage. The decompression audit further shows that LogNexus successfully restores every token on all 16 datasets.
Article Search Artifacts Available Article: issta26main-p1971-p (type: Full Paper) doi:10.1145/3832263

Artifact for LogNexus: Effective Log Compression via Unified Redundancy Encoding (doi:10.5281/zenodo.21281836): Artifact for the ISSTA 2026 paper “LogNexus: Effective Log Compression via Unified Redundancy Encoding.” The archived package contains the LogNexus source code, a prebuilt linux/amd64 Docker image, documentation, a reduced public dataset, experiment scripts, configuration files, and reference result snapshots. It ...

Paired Code Smells and Test Smells: A Fine-Grained Longitudinal Empirical Study
Ziwen Cai
(Chinese University of Hong Kong, Shenzhen, China)
As software systems grow in complexity, ensuring maintainability is essential, but often hindered by various quality issues. Among them, code smells and test smells are widely recognized indicators of technical debt that degrade system quality. While production code and test suites are intrinsically coupled and evolve together, existing research predominantly studies code smells and test smells in isolation, ignoring their underlying connection over time.
To bridge this gap, this paper presents a fine-grained longitudinal study on the co-evolutionary dynamics of "paired smells", which refer to cases where code and test smells exist concurrently in linked production and test code. By analyzing 128,417 commits across 19 long-lived open-source Java repositories, we investigate the statistical associations, survival lifespans, and removal motivations of these paired flaws. As a result, we identify 30 statistically significant rules where specific test smells imply the presence of underlying production code smells and our survival analysis reveals that in most repositories, paired smells have relatively shorter lifespans than unpaired ones. To explain this phenomenon, our manual inspection reveals that paired smells often signal deeper structural issues, prompting explicit risk-driven refactoring. Finally, we discuss the implications of these results for code quality analysis and continuous integration tools, suggesting that they should link related smell findings between production and test code and guide synchronized refactoring.
Article Search Article: issta26main-p1977-p (type: Full Paper) doi:10.1145/3832264

Fixed-Point Guided ADS Scenario Generation via Multi-modal LLM Reasoning and Software Testing
Xudong Zhang, Shihao Zhu, and Yan Cai
(Institute of Software at Chinese Academy of Sciences, China)
Robust scenario generation is essential for systematically testing Autonomous Driving Systems (ADSs) under rare and safety-critical conditions. However, search-based approaches often lack semantic guidance, whereas specification-based approaches rely heavily on manually constructed rules. Existing LLM-assisted techniques primarily translate accident artifacts into scene descriptions without producing executable behavioral specifications that guide subsequent testing. We present InvarGen, a framework that uses a multi-modal LLM as a parametric specification generator. InvarGen organizes safety requirements into a predefined taxonomy of Scenario Fixed Points while dynamically instantiating their predicates, thresholds, and temporal bounds from each accident context. These fixed points serve as executable test oracles and optimization objectives. Unlike static templates, these fixed points adaptively constrain the search space, guiding a hybrid evolutionary process: Intelligent Fuzzing exploits boundary parameters to trigger specific violations, while Structural Mutation ensures the global exploration of diverse environmental contexts. Evaluation using 200 real-world accidents and 1,400 synthesized scenarios shows that InvarGen discovers 37.8% more critical scenario types than the best baseline, achieves a 30% fixed-point violation rate, and achieves the highest semantic-diversity score. The scenarios also achieve high physical plausibility and cross-simulator executability, with 100% syntactic compliance with the evaluated OpenX formats. These results highlight the promise of fixed-point semantics as a principled bridge between unstructured LLM reasoning and rigorous robustness testing.
Article Search Artifacts Available Article: issta26main-p2001-p (type: Full Paper) doi:10.1145/3832265

Reproduction package for article "Fixed-Point Guided ADS Scenario Generation via Multi-modal LLM Reasoning and Software Testing" (doi:10.5281/zenodo.21817327): The artifact accompanying this paper is permanently archived on Zenodo at \url{https://doi.org/10.5281/zenodo.21817327}. The package includes the implementation, DSL grammar, prompts and configurations, redistributable data or metadata for the test cases and generated scenarios. And the INVARGEN is a novel framework ...

ARQ: A Mixed-Precision Quantization Framework for Accurate and Certifiably Robust DNNs
Yuchen Yang, Yifan Zhao, Shubham Ugare, Gagandeep Singh, and Sasa Misailovic
(University of Illinois at Urbana-Champaign, USA)
Mixed precision quantization has become an important technique for optimizing the execution of deep neural networks (DNNs). Certified robustness, which provides provable guarantees about a model’s ability to withstand different adversarial perturbations, has rarely been addressed in quantization due to the unacceptably high cost of certifying robustness. This paper introduces ARQ, an innovative mixed-precision quantization method that not only preserves the clean accuracy of the smoothed classifiers, but also maintains their certified robustness. ARQ uses reinforcement learning to find accurate and robust DNN quantization, while efficiently leveraging randomized smoothing, a popular class of statistical DNN verification algorithms. ARQ consistently performs better than multiple state-of-the-art quantization techniques across all the benchmarks and the input perturbation levels. The performance of ARQ quantized networks reaches that of the original DNN with floating-point weights, while using only 1.5% instructions and the highest certified radius. ARQ’s code is available at https://github.com/uiuc-arc/ARQ.
Article Search Article: issta26main-p2010-p (type: Full Paper) doi:10.1145/3832266

Red-Teaming Coding Agents from a Tool-Invocation Perspective: An Empirical Security Assessment
Yuchong Xie, Mingyu Luo, Zesen Liu, Zhixiang Zhang, Kaikai Zhang, Yu Liu, Ci Tao, Changhui Wang, Zongjie Li, Ping Chen, Shuai Wang, and Dongdong She
(Hong Kong University of Science and Technology, Hong Kong; Fudan University, China)
Coding agents powered by large language models are becoming central modules of modern IDEs. They help users to perform various complex coding tasks by invoking tools. Although powerful, tool-invocation operation in coding agents opens a substantial attack surface for adversaries. Prior work has demonstrated attacks against both general-purpose LLM agents and domain-specific agents. However, to our knowledge, no previous works focus on the security risks of tool-invocation in coding agents. To fill this gap, we conduct the first systematic, in-depth red-teaming from a tool-invocation perspective in six popular real-world coding agents: Cursor, Claude Code, Copilot, Windsurf, Cline, and Trae.
Our red-teaming proceeds in two phases. In Phase 1, we conduct a prompt leakage as reconnaissance to recover system prompts and related context. Specifically, we identify a mode gap between chat generation and tool-call argument generation: during schema-driven argument completion, the model behaves as if it is performing benign structured filling and may copy hidden agent context into tool-call arguments. We instantiate this gap as ToolLeak, which exfiltrates agent-internal prompts (e.g., system prompts and tool metadata) via required tool parameters. In Phase 2, we hijack the tool-invocation behavior of the coding agent with a novel two-channel prompt injection in the tool description and the tool return. Our hijacking achieves remote code execution (RCE) on major real-world coding agents. We adaptively construct the malicious payload using leaked security information in Phase 1.
Our evaluation shows that ToolLeak substantially outperforms strong prompt-leak baselines in both emulated and real-world settings. In the emulated setting, ToolLeak achieves the best overall prompt-exfiltration performance across all six simulated coding agents. On real-world coding agents, ToolLeak achieves the best pseudo-recall on 18 of 25 evaluated agent-LLM pairs. Furthermore, our red-teaming successfully hijacks all six real-world coding agents for RCE and consistently yields higher attack success rates than baseline attacks. Lastly, we present two case studies on Cursor and Claude Code to demonstrate the real-world impact of our red-teaming.
Article Search Article: issta26main-p2031-p (type: Full Paper) doi:10.1145/3832267

How Safe Is Your Screen? Understanding and Detecting Privacy Leaks in Sensitive Activities
Youngseok Kim, Sungho Lee, and Sungjae Hwang
(Sungkyunkwan University, Republic of Korea; Chungnam National University, Republic of Korea)
Android apps handle sensitive data, often displayed on screens (i.e., Activities), and protecting these activities is vital for ensuring user privacy. Several studies have demonstrated the risk of private data leakage via screen capture and identified malware exploiting this vulnerability. To mitigate such threats, Google introduced FLAG_SECURE, an activity-level security feature that protects an Activity when set by blocking screen capture and restricting display on non-secure devices. Proper usage of this security feature could significantly reduce data leakage risks. However, to the best of our knowledge, its adoption across diverse Android apps has not been systematically studied.
In this work, we present a large-scale empirical study of screenshot-based vulnerabilities in Android apps using a systematic static analysis pipeline, ASSA. The pipeline automatically constructs UI models of activities and fragments by leveraging a customized version of Frontmatter, and combines them with a Large Language Model (LLM) to infer runtime context and identify sensitive user data rendered on screens. We apply ASSA to 5,667 popular real-world Android apps and find that 19.8% contain at least one activity that displays sensitive information without screen-capture protection, while 87.2% of apps do not apply FLAG_SECURE to any activity, exposing credentials, financial information, and location data to screenshot attacks. Beyond quantifying prevalence, we analyze FLAG_SECURE usage patterns and establish a taxonomy that captures diverse factors influencing developers’ decisions to adopt or omit screen protection, including intentional privacy protection, intellectual property safeguarding, and unintentional inheritance from base activities. To assess real-world impact, we further conduct in-depth case studies on 200 vulnerable apps and responsibly disclose our findings to the affected vendors, receiving 6 confirmed patches and 18 acknowledgments, including 10 vendors who reported ongoing efforts toward mitigation. These findings indicate that screenshot-based vulnerabilities are not only widespread but also systematically overlooked in practice, highlighting the need for greater awareness and more careful handling of sensitive on-screen information by developers.
Article Search Artifacts Available Article: issta26main-p2055-p (type: Full Paper) doi:10.1145/3832268

Source Code of ASSA tool from Article 'How Safe Is Your Screen? Understanding and Detecting Privacy Leaks in Sensitive Activities' (doi:10.6084/m9.figshare.30082021.v3): Purpose: ASSA (Android Screen Sensitivity Analyzer) is a static analysis tool designed to detect Android applications vulnerable to screenshot and screen-recording privacy leaks. This artifact accompanies the ISSTA 2026 paper "How Safe Is Your Screen? Understanding and Detecting Privacy Leaks in Sensitive Activities" ...

Cracking Query Bottlenecks: Towards Efficiency-Oriented Text-to-SQL Generation
Li Lin, Yunfeng Shen, Lingfeng Bao, Rongxin Wu, and Yang Liu
(Zhejiang University, China; Xiamen University, China; Nanyang Technological University, Singapore)
Text-to-SQL models translate natural language questions into SQL, enabling non-technical users to access databases. However, most existing research focuses on correctness, neglecting query efficiency. In this paper, we address the challenge of evaluating the execution efficiency of generated SQL in Text-to-SQL by introducing EESQLBench, a novel benchmark designed to assess both correctness and efficiency. EESQLBench pairs each natural language question with an expert-optimized SQL query, providing a reliable efficiency baseline. We evaluate six representative large language models (LLMs), including four open-source models (SQLCoder, CodeLlama, DeepSeek-Coder, and DeepSeek-R1) and two closed-source models (GPT-5.2 and Gemini-2.5-Pro), using cost-based metrics including Cost Reachability (CR) and Acceptable Reachability at kk (AR@kk). Our results reveal that current LLMs, despite achieving high correctness, struggle to produce efficient queries. We observe substantial efficiency gaps between models and emphasize that semantic correctness alone does not guarantee query efficiency. Furthermore, we provide insights into common inefficiency patterns in LLM-generated SQL queries, such as missing access pruning and inefficient subquery logic.
Article Search Article: issta26main-p2080-p (type: Full Paper) doi:10.1145/3832269

Lingxi: Repository-Level Issue Resolution Framework Enhanced by Procedural Knowledge Guided Scaling
Xu Yang, Jiayuan Zhou, Michael Pacheco, Wenhan Zhu, Pengfei He, Shaowei Wang, Kui Liu, and Ruiqi Pan
(University of Manitoba, Canada; Huawei, Canada; Huawei Technologies, China)
Driven by the advancements of Large Language Models (LLMs), LLM-powered agents are making significant improvements in software engineering tasks, yet struggle with complex, repository-level issue resolution. Existing agent-based methods have two key limitations. First, they lack of procedural knowledge (i.e., how an issue is fixed step-by-step and rationales behind it) to learn and leverage for issue resolution. Second, they rely on massive computational power to blindly explore the solution space. To address those limitations, we propose Lingxi, an issue resolution framework that leverages procedural knowledge extracted from historical issue-fixing data to guide agents in solving repository-level issues. Lingxi first constructs this knowledge offline through a hierarchical abstraction mechanism, enabling agents to learn the how and why behind a fix, not just the final solution. During online application, it employs a knowledge-driven scaling method that leverages the procedural knowledge of similar issues to intelligently analyze the target issue from multiple perspectives, in sharp contrast to undirected, brute-force exploration.Lingxi successfully resolves 74.6% of bugs on the SWE-bench Verified benchmark in Pass@1 setting, outperforming five state-of-the-art techniques by a significant margin (5.4% to 14.9%). Our comprehensive ablation study confirmed that the success of Lingxi comes directly from its use of procedural knowledge. Without it, the performance gains from scaling alone is negligible. Our qualitative study further shows that the “design patterns & coding practices” is the most critical knowledge aspect, and that the roles of different knowledge aspects switch across different stages (i.e., analysis, planning, and fixing).
Article Search Article: issta26main-p2081-p (type: Full Paper) doi:10.1145/3832270

Characterizing Real-World Bugs in Tile Programs for Automated Bug Detection
Ravishka Rathnasuriya, Zihe Song, Nidhi Majoju, Tingxi Li, Aaryaa Moharir, Wei Yang, and Tao Xie
(University of Texas at Dallas, USA; Peking University, China)
Tile-based programming frameworks are increasingly adopted to write high-performance GPU kernels in domains such as deep learning and scientific computing. While these frameworks enhance productivity and hardware utilization, their multi-stage compilation pipelines introduce distinct code generation bugs that are tightly coupled to input shapes, data types, and backend targets. These bugs often manifest as silent wrong results or performance issues, making them difficult to detect using existing compiler testing tools. Additionally, the unique programming conventions of tile domain-specific languages complicate root cause identification, while fixing such bugs demands specialized knowledge of tile abstractions and compilation pipelines. Despite the growing adoption of tile-based systems, their code generation bugs remain largely unexplored. This paper presents the first systematic study of tile-program code generation bugs. We curate 401 bug reports from GitHub and identify 301 tile-program codegen bugs for analysis, characterizing their root causes and symptoms, the input patterns that trigger them, the test oracles that detect them, and the strategies for fixing these bugs. Our study provides foundational insights for building debugging, testing, and repair tools tailored to tile-based compiler infrastructures.
Article Search Article: issta26main-p2094-p (type: Full Paper) doi:10.1145/3832271

Automated Program Repair for UI-Centric Android Bugs: How Far Are We?
Junayed Mahmud, Sparsh Pandey, Nadeeshan De Silva, Atish Kumar Dipongkor, Jingjing Wu, Oscar Chaparro, Mattia Fazzini, and Kevin Moran
(University of Central Florida, USA; William & Mary, USA; University of Minnesota, USA)
Substantial research effort has been devoted to developing techniques for automated program repair (APR) that suggest patches for localized buggy code -- and more recent techniques have begun to leverage the capabilities of code-centric large language models (LLMs). However, the scope and diversity of bugs to which these techniques have historically been applied are limited. In particular, the research community currently lacks a comprehensive understanding of the performance of APR techniques on bugs that arise in UI-centric programs, such as mobile apps. Bugs in UI centric programs carry with them unique challenges, including (i) the need to reason across interconnected subroutines that connect presentation and program logic, (ii) event-driven programming paradigms, and (iii) the need to reason about program state through cues in the UI.
In this paper, we investigate the effectiveness of existing APR techniques when applied to fix bugs in UI-centric programs - specifically Android applications. To explore this phenomenon, we conduct a comprehensive empirical study with five existing program repair techniques (including those that utilize LLMs) on a hybrid dataset including 46 synthetic bugs, generated via MDroid+, an Android-specific mutation tool, and 50 real bugs systematically mined from issue reports of 23 popular Android applications. Our findings illustrate important current limitations in resolving UI-related issues in mobile apps. We synthesize these results to form a taxonomy of the limitations of existing program repair techniques. This taxonomy outlines key limitations and can inform future research efforts in designing automated program repair tools for UI-centric bugs in mobile applications.
Article Search Article: issta26main-p2095-p (type: Full Paper) doi:10.1145/3832272

EventSpec: Defining and Detecting Event-Semantic Issues in Blockchain Ecosystems
Yixuan Liu, Yuxin Dong, Ye Liu, Yin Wu, Chengxuan Zhang, Xiapu Luo, and Yi Li
(Nanyang Technological University, Singapore; Peking University, China; Beijing Institute of Technology, China; Xi'an Jiaotong University, China; Hong Kong Polytechnic University, China)
In recent years, smart contracts have become the backbone of decentralized applications (DApps), and off-chain systems such as bridges, wallets, and indexers rely heavily on event logs to track contract execution and state changes. However, the Ethereum Virtual Machine (EVM) does not validate or enforce event semantics, so logs can diverge from on-chain state, misleading off-chain systems into accepting incorrect state transitions. Existing smart contract vulnerability detection tools focus on logic bugs, with limited support for detecting event-semantic defects. To address this gap, we collect audit reports and incident cases and apply open card sorting to define five classes of event-semantic defects: event collision, state-event mismatch, unauthorized event emission, event emission mismatch, and event parameter mismatch. We propose EventSpec, which infers event specifications from a contract corpus via behavior inference and semantic-constraint extraction and applies differential checking to identify event-semantic defects in target contracts. We run EventSpec on 6,617 real-world contracts and evaluate detection effectiveness based on manually labeled results; EventSpec achieves an overall comprehensive precision of 90.17%. We further provide an off-chain evaluation harness that reproduces two off-chain attack vectors on any EVM-compatible chain: event origin confusion caused by unintended emitters and event–state desynchronization where events lack matching state updates. Using this harness, we demonstrate the feasibility of these attacks on bridge relayers, blockchain explorers, and NFT marketplaces, and report six wallet issues, four of which were confirmed (including a $600 bounty), with two remaining pending.
Article Search Article: issta26main-p2097-p (type: Full Paper) doi:10.1145/3832273

avaCGs: Version-Aware Call Graphs for Efficient Version-Range Queries
Johannes Düsing, Dominik Helm, and Ben Hermann
(University of Stuttgart, Germany)
Developers employ version ranges to specify a range of valid versions for software libraries their projects depend on. While this can yield benefits like automatic adoption of library updates, it complicates method reachability analysis: a sound whole-program analysis must consider method invocations of every library release within that range. Such releases might themselves introduce transitive ranged dependencies to a project, leading to a combinatorial blow-up in the number of configurations to analyze. If developers wanted to soundly determine whether a critical method might be reachable via a ranged library dependency, they would have to build the individual call graphs for every release within that range, and perform reachability analysis on each one of them. As call-graph construction is an expensive operation, this approach is rarely practical. To enable direct version-range queries, we introduce artifact version-aware call graphs (avaCGs) that comprise call graph information about all versions of a software artifact in a single graph structure. Further, we propose a novel approach that incrementally computes call graphs based on Rapid Type Analysis (RTA), implement it for the JVM platform, and show that it yields identical results to full RTA call-graph builds. Our evaluation shows that on our benchmarks, avaCGs can improve the performance of reachability queries by up to 9.58x, while our incremental construction is up to 79% faster compared to full builds for each release. We also observe that for real-world libraries hosted on Maven Central, almost 80% of all releases do not change the RTA call graph compared to their previous release, further justifying the use of incremental approaches.
Article Search Artifacts Available Article: issta26main-p2099-p (type: Full Paper) doi:10.1145/3832274

Companion Artifact for "avaCGs: Version-Aware Call Graphs for Efficient Version-Range Queries" (doi:10.5281/zenodo.20815429): This artifact contains our prototype for incremental construction of artifact version-aware call graphs (avaCGs) of JVM libraries hosted on Maven Central. Alongside we provide all data used in the evaluation of our publication "avaCGs: Version-Aware Call Graphs for Efficient Version-Range Queries", which has been ...

What Makes In-Context Examples Effective for Code Generation?
Dongze Li, Songqiang Chen, Jialun Cao, and Shing-Chi Cheung
(Hong Kong University of Science and Technology, China; Guangzhou HKUST Fok Ying Tung Research Institute, China)
In-Context Learning (ICL) has emerged as a promising solution to enhance the code generation capabilities of Large Language Models (LLMs) by incorporating code examples inside the prompt to let LLMs learn from demonstrations. However, despite their effectiveness gains, it remains unclear which specific properties of ICL-provided code examples (e.g., solution insight, essential contextual information, identifier naming styles, code formatting) drive these gains. This paper systematically investigates the impact of different sources and internal features of code examples on ICL for code generation through controlled experiments on contest-style programming questions and repository-level tasks. Our results show that while LLMs struggle to extract generalizable problem-solving insights from provided solutions to similar questions or repository snippets, their retrieval-augmented ICL performance can significantly benefit from explicit contextual information, such as input/output demonstrations, required helper functions, and namespace information. Through targeted mutation operators, we further find that identifier naming is substantially more critical than code formatting or low-level implementation details, with the elimination of descriptive variable names causing performance drops of up to 30 percentage points. Finally, we demonstrate that LLMs significantly prefer semantically meaningful identifier names and that adherence to surface-level naming conventions is far less important than semantic clarity. These findings provide practical guidelines for constructing effective ICL code examples and highlight challenges in reflection-based learning for code generation.
Article Search Artifacts Available Article: issta26main-p2106-p (type: Full Paper) doi:10.1145/3832275

What Makes In-Context Examples Effective for Code Generation? (Artifact) (doi:10.6084/m9.figshare.32835674.v2): This artifact provides the replication code, as well as the experiment results reported in the tables from the paper: What Makes In-Context Examples Effective for Code Generation?.

Code-MUE: Measuring Code LLMs’ Uncertainty through Execution-Based Semantic Interaction Graphs
Xiaoning Ren, Yinxing Xue, Lei Ma, and Yuheng Huang
(Xi’an Jiaotong University, China; Institute of AI for Industries at Chinese Academy of Sciences, China; University of Tokyo, Japan; University of Alberta, Canada)
As Code Large Language Models (LLMs) become central to modern software engineering, their inherent stochasticity poses significant real-world risks, where even minor errors can lead to severe functional, security, or safety consequences. Reliable automation, therefore, demands the ability to distinguish between confident, well-supported predictions and stochastic guessing. However, existing uncertainty estimation methods face a critical gap: white-box and grey-box techniques are often inapplicable to closed-source models, while standard black-box text metrics fail to capture the unique fragility of code, where syntactic variation does not necessarily imply semantic divergence. To bridge this syntax-semantics gap, we introduce Code-MUE, a purely black-box framework that measures uncertainty through execution-based Semantic Interaction Graphs. Unlike prior approaches that rely on superficial textual similarity, Code-MUE grounds uncertainty in observable runtime behavior and calculates the von Neumann entropy of the solution space to quantify global semantic diversity. A large-scale empirical study across eight state-of-the-art LLMs demonstrates that Code-MUE achieves a strong negative correlation with functional correctness, with Spearman’s correlation reaching up to −0.98. It significantly outperforms lexical and embedding-based baselines while enabling robust risk detection and selective prediction in practical workflows.
Article Search Article: issta26main-p2153-p (type: Full Paper) doi:10.1145/3832276

RICE: Harnessing LLMs and Historical Issues to Discover Internal Rust Compiler Errors
Langyi Lu, Wei You, Bin Liang, and Jianjun Huang
(Renmin University of China, China)
Rust is a modern system-level programming language that emphasizes safety, concurrency, and performance. Ensuring the reliability of the Rust compiler is critical, as undetected compiler defects, particularly internal compiler errors (ICEs), can lead to runtime crashes or undefined behavior in system software. Existing compiler testing methods, including generation-based, mutation-based, and recent LLM-driven approaches, are limited in their ability to systematically uncover ICEs triggered by advanced language features or complex code interactions. In this paper, we propose RICE, an LLM-assisted Rust compiler testing framework that generates test cases by adapting defect-prone patterns extracted from historical ICE issues to diverse code contexts in the official test suite. RICE features simplification-guided identification of defect-prone patterns and context- aware adaptation of patterns to maintain compilability and meaningful semantic interactions. Applied to multiple recent Rust compiler releases, RICE discovered 37 ICEs, including 15 previously unknown zero-day bugs, outperforming existing testing tools in code coverage and ICE discovery. Our results demonstrate that systematically leveraging historical defect-prone patterns enables more effective testing, accelerates bug fixing, and enhances regression testing in Rust compilers
Article Search Article: issta26main-p2212-p (type: Full Paper) doi:10.1145/3832277

Efficient Predictive Monitoring of Message Passing Interface Programs
Jiaqiang Yao, Haocheng Geng, and Zhenbang Chen
(National University of Defense Technology, China)
The Message Passing Interface (MPI) is the standard programming model for high-performance computing, yet nondeterministic scheduling in concurrent executions makes reliability assurance highly challenging. Beyond deadlocks, property-related bugs such as modifying a send buffer before a nonblocking send completes or accessing a freed RMA window are often hard to reproduce and validate, as they typically manifest only under rare event interleavings and can be both latent and catastrophic. Existing dynamic checkers usually cover only the observed schedule of a single execution; dynamic verification largely focuses on deadlocks; and static techniques scale poorly to medium-to-large programs, often timing out or running out of memory. Overall, existing techniques do not support scalable analysis of temporal properties in large MPI programs.
To address these limitations, we propose the first efficient predictive monitoring approach for temporal properties in MPI programs. From a single execution, we collect an event trace and use trace equivalence to predict the set of legal equivalent executions under the same input. We then check whether any predicted execution satisfies the target property. To make this process sound, we formalize three correctness criteria for MPI trace reordering and enforce them through MPI-semantics-driven dependency extraction, tracked with vector clocks. To improve efficiency, we exploit the bounded length of a pattern language and reduce long-trace reordering to reordering short patterns. We implement our approach in MPI-PRV, instantiate ten representative MPI bug properties, and evaluate it on 13 real-world MPI applications with 38 configurations. MPI-PRV successfully and correctly analyzes all 38 tasks, whereas MPI-SV and MUST analyze only 14 and 11 tasks, respectively. MPI-PRV also achieves orders-of-magnitude reductions in runtime and memory usage, demonstrating strong efficiency and scalability for large MPI programs.
Article Search Article: issta26main-p2224-p (type: Full Paper) doi:10.1145/3832278

Contamination Means Overestimation? A Fine-Grained Empirical Study in Code Intelligence
Zhen Yang, Hongyi Lin, Yifan He, Junqi Wang, Zeyu Sun, Shuo Liu, Jie Xu, Pengpeng Wang, Zhongxing Yu, and Qingyuan Liang
(Shandong University, China; Institute of Software at Chinese Academy of Sciences, China; City University of Hong Kong, Hong Kong; Columbia University, USA; Peking University, China)
In recent years, code intelligence has gained increasing importance in the field of automated software engineering. Meanwhile, the widespread adoption of Pretrained Language Models (PLMs) and Large Language Models (LLMs) has raised concerns regarding data contamination and its potential impact on model performance evaluation. Previous studies mainly focused on sample-level contamination, ignoring partial contamination scenarios that are pervasive in code intelligence. This paper fills this gap and presents a systematic empirical study to investigate the fine-grained data contamination on mainstream code tasks. Our study involves diverse representative PLMs: RoBERTa and GPT-2, and LLMs: LLaMA and StarCoder, covering three major tasks: code translation, code generation, and code summarization, across two Programming Languages (PLs): Java and Python. We categorize contamination scenarios into four types according to the code intelligence practice, namely input-only, output-only, unpaired, and paired contamination settings, and construct corresponding experimental and control groups for exploration.
Experimental results show that, under the pre-training, fine-tuning, and inference paradigm adopted by PLMs, even deliberately injecting paired contamination does not lead to significant performance overestimation. But direct inference or small-scale fine-tuning uncovers the contamination effects. In contrast, LLMs with pre-training and inference paradigm are significantly affected by the paired contamination. Apart from the above, other contamination scenarios have no impact on both PLMs and LLMs. Our findings challenge the conventional belief that contamination inevitably leads to performance overestimation, providing new insights into the evaluation and deployment of code intelligence models.
Article Search Article: issta26main-p2238-p (type: Full Paper) doi:10.1145/3832279

A Closer Look at the Use of Reinforcement Learning for Speeding Up Runtime Verification of Software Tests (Experience Paper)
Shinhae Kim, Saikat Dutta, and Owolabi Legunsen
(Cornell University, USA)
Runtime verification (RV) found many bugs by monitoring passing tests against formal specifications (specs), but it is slow. A recent work, Valg, used reinforcement learning (RL) to speed up RV by up to 551.5x or 27 hours. Valg aims to probabilistically monitor most unique traces—sequences of spec-related events like method calls—and monitor fewer redundant ones. But, there is no in-depth study of Valg’s current limits and how to address them. We study Valg on 93 Java open-source projects to answer five unaddressed questions. (i) How much slower is Valg than optimal baselines? Up to 323.8x, or 3.2 hours vs. running tests without RV, and up to 9.6x, or 25.3 minutes vs. a theoretically optimal baseline that monitors only unique traces. (ii) Where is Valg’s time spent? 30.5% on monitoring and 18.4% on signaling events to monitors, on average. (iii) What characterizes code where Valg monitors too many redundant traces or misses unique ones? In 100 cases, 67.8% of redundant traces are due to limitations of Valg’s RL convergence heuristic, and 41.3% of missed unique traces occur when a Valg assumption does not hold. (iv) How much can test non-determinism and RL stochasticity cause monitored unique traces to vary? By 42.5 percentage points (pp) and 12.3pp on average, respectively, but they vary by up to 98pp. (v) How do other off-the-shelf RL algorithms compare with Valg’s? Only two of 11 RL algorithms that we survey are feasible for RV during continuous integration. Both are slower and miss more unique traces than Valg, so custom RL algorithms for RV may be needed. So, despite Valg’s promising results, it has plenty of room to improve. We highlight several exciting future directions on using RL to speed up RV.
Article Search Article: issta26main-p2242-p (type: Full Paper) doi:10.1145/3832280

WITFuzz: Validity-Preserving Greybox Fuzzing for WebAssembly Interface Type Binding Generators
Hanqin Guan, Ningyu He, Shangtong Cao, Yifeng Cai, Yao Guo, and Ding Li
(Peking University, China; Hong Kong Polytechnic University, Hong Kong; Beijing University of Posts and Telecommunications, China; Beijing Tongming Lake Information Technology Application Innovation Center, China)
Modern build pipelines often rely on code generation to turn constraint-rich interface specifications into artifacts for target programming languages. In the WebAssembly component model, binding generators (bindgens) follow this pattern by translating WebAssembly Interface Types (WIT) packages into language-specific bindings that are later compiled with application code. This bindgen step already targets more than ten language ecosystems, and the Rust wit-bindgen crate alone has accumulated tens of millions of downloads. Yet a WIT package may pass WIT validation but still break this build pipeline: bindgens may crash or hang during generation (Phase I), or downstream toolchains may reject the generated bindings even when generation succeeds (Phase II). Testing bindgens at scale is challenging because WIT is strongly typed and constraint-rich, and Phase II failures require language-specific checking. We present WITFuzz, a validity-preserving greybox fuzzer for WIT bindgens. WITFuzz mutates resolved WIT abstract syntax trees via structure-aware rewrites expressed in a small domain-specific language, and propagates correlated updates to maintain WIT validity. When coverage plateaus, WITFuzz expands its strategy pool online using coverage-guided, LLM-assisted DSL synthesis, admitting only strategies that pass local validation. WITFuzz further uses a build-aware, multi-layer oracle that combines in-loop checks with selective asynchronous compilation/typechecking of generated bindings to capture non-crashing build breakers. Across 12 bindgens, WITFuzz improves average edge coverage by 8.3% over standalone wit-smith. It uncovers 40 previously unknown Phase I and Phase II build-breaking bugs, including 35 that are missed by all external baselines.
Article Search Artifacts Available Article: issta26main-p2263-p (type: Full Paper) doi:10.1145/3832281

WITFuzz: Validity-Preserving Greybox Fuzzing for WebAssembly Interface Type Binding Generators (Artifact) (doi:10.5281/zenodo.21066459): This archive contains the WITFuzz source tree, Docker packaging, scripts, WIT seeds, reproducers, and sample summaries.

On the Evaluation of Large Language Models in Unit Test Evolution (Experience Paper)
Weichang Liu, Junwei Zhang, Yuqing Niu, and Bo Zhou
(Zhejiang University, China; Singapore Management University, Singapore)
Large language models (LLMs) have recently shown promising potential in automating unit test evolution for evolving software systems. However, the effectiveness of LLMs in unit test evolution remains insufficiently understood, particularly with respect to prompt design choices, in-context learning (ICL) strategies, and different types of test evolution. In this paper, we present the first comprehensive empirical study to evaluate LLMs for unit test evolution. We systematically assess nine open-source code LLMs (3B to 34B parameters) and three state-of-the-art commercial models across diverse prompt designs, ICL strategies, and representative test evolution frameworks. To support robust and execution-based evaluation, we construct a new benchmark consisting of 530 real-world focal method–test co-evolution instances collected from seven actively maintained open-source projects. Our evaluation employs a suite of compilation, execution, and coverage-based metrics. Extensive experimental results reveal that prompt design and ICL methods significantly impact LLM effectiveness. Furthermore, the optimal configurations of these strategies vary substantially across different LLMs and evolution types. Based on our findings, we derive actionable insights to guide future research and practical adoption of LLM-based techniques for unit test evolution.
Article Search Artifacts Available Article: issta26main-p2286-p (type: Full Paper) doi:10.1145/3832282

Reproduction Package for Article `On the Evaluation of Large Language Models in Unit Test Evolution (Experience Paper)` (doi:10.5281/zenodo.21696965): This artifact serves as a comprehensive replication package for the associated paper. It comprises the complete source code, necessary environment configurations, and the specific datasets. The primary objective of this package is to enable researchers to seamlessly replicate our experiments and utilize the ...

STARS: Static Analysis-Guided Assertion Synthesis using Large Language Models
Jialun Cao, Haoyu Wang, Haoran Yan, Ming Wen, and Michael Pradel
(Hong Kong University of Science and Technology, China; Guangzhou HKUST Fok Ying Tung Research Institute, China; Huazhong University of Science and Technology, China; CISPA Helmholtz Center for Information Security, Germany)
Automated unit test generation promises to reduce the cost of software quality assurance, and hence, is attracting attention from both academia and industry. Yet, generating assertions that are executable, meaningful to developers, and able to catch faults remains an unsolved challenge. Existing approaches either randomly enumerate assertions that are plausible based on static program analysis without considering whether they naturally fit the test prefix or query an LLM to generate assertions based on local context only, such as the test prefix and the focal method. However, we observe that local context alone is insufficient for LLMs to generate high-quality assertions because many desirable assertions are built from components that are almost impossible to guess for an LLM, such as sequences of multiple method calls. This paper presents STARS, a novel test assertion generation technique that combines the benefits of static program analysis and LLM-based synthesis. The key idea is to first gather a set of assertion components based on static program analysis and to then combine, concretize, prioritize, and improve them with an LLM. The resulting assertions go beyond what an LLM alone could realistically guess based on the test prefix and focal method, and they naturally fit the given test case. Empirical results show that STARS consistently outperforms the state-of-the-art baseline in five evaluation metrics. STARS achieves an exact-match rate of at most 83.4% using GPT-5.4. Compared with the baseline, STARS’s mutation score nearly triples that of the baseline (10.97% vs. 3.97%), approaching that of developer-written assertions, while consuming 24.75% fewer tokens and 30.6% fewer LLM queries.
Article Search Artifacts Available Article: issta26main-p2323-p (type: Full Paper) doi:10.1145/3832283

Reproduction Package for the Article "STARS: Static Analysis-Guided Assertion Synthesis using Large Language Models" (doi:10.5281): The artifact includes the source code to reproduce the results, dataset for experiment, and experiment results.

AdaptAgent: A Multi-agent, Domain-Guided Reasoning Framework for Code Adaptation
Xiaokai Rong, Hridya Dhulipala, Aashish Yadavally, and Tien N. Nguyen
(University of Texas at Dallas, USA; University of Central Florida, USA)
Developers often need to adapt into their projects the code generated from LLMs or code snippets from online forums. However, integrating them into an existing repository remains challenging in a manual process. A successful integration typically requires more than copying code as a user must produce correct adapting changes at a designated location in the target repository. We formalize this as the code adaptation problem: given a snippet, functional intent, a target repository, and an adaptation location, generate a patch that adapts the snippet into the repository. We present AdaptAgent, a multi-agent, domain-guided reasoning framework for code adaptation. Rather than relying on single-shot prompting, AdaptAgent decomposes adaptation into specialized agents that communicate via typed artifacts: an Intent Summarizer extracts adaptation goals from Q&A text; a Policy Agent derives domain policies from six adaptation categories; a Domain Planner generates a self-ordered plan; a Context Miner distills sibling-method semantics from the target codebase; and a Code Adapter realizes the plan as a minimal unified diff, iteratively refined using a compiler-based Verifier. This division of labor enables robust, policy-aligned adaptations and supports adapting code snippets into a project. On a real-world dataset, AdaptAgent outperforms strong baselines in semantic correctness and produces patches that mirror developers’ actual adaptation patterns. Our ablation study shows each agent’s necessity, especially planning for code-hardening and exception-handling, and intent for logic customization.
Article Search Article: issta26main-p2324-p (type: Full Paper) doi:10.1145/3832284

Understanding Automated Program Repair Agents through the Lens of Traceability: An Empirical Study
Ira Ceka, Hailie Mitchell, Saurabh Pujar, Luca Buratti, Shyam Ramji, Junfeng Yang, Gail Kaiser, and Baishakhi Ray
(Columbia University, USA; IBM Research, USA)
Automated Program Repair (APR) agents leverage large language models (LLMs) to autonomously diagnose and patch software bugs using planning, reasoning, and tools. Although these agents show strong performance on leaderboards such as SWE-bench, little is understood about how they take actions, where they fail, and how their behavior compares to human developers. In this paper, we present the first systematic analysis of these limitations using 5 state-of-the-art APR agents. We trace the full decision-making pipelines of the 5 APR agents across 500 real-world repair tasks, from issue description to patch validation.
Our study reveals that, while agents excel at simple fixes, they struggle with logic-intensive bugs, often generating verbose, overfitted patches that pass existing test suites without solving the root cause. Test generation and regression test selection remain major bottlenecks, as agents fail to reproduce issues or run relevant regression tests. Moreover, many agents operate with primitive tooling (e.g. bash scripts) and do not have access to debuggers or program analysis tools. These findings highlight key limitations of current APR systems and motivate several directions for next-generation APR design, including but not limited to: (1) a shift-left approach emphasizing early, high-quality test generation and validation to reduce spurious fixes and improve semantic correctness; (2) richer, more integrated tool ecosystems; (3) diversified agent architectures that combine complementary strengths; and (4) benchmarks that prioritize semantic repair quality and test-generation fidelity over surface-level success metrics.
Article Search Article: issta26main-p2374-p (type: Full Paper) doi:10.1145/3832285

CONCUR: Benchmarking LLMs for Concurrent Code Generation
Jue Huang, Tarek Mahmud, Corina S. Pasareanu, and Guowei Yang
(University of Queensland, Australia; Texas A&M University, Kingsville, USA; Carnegie Mellon University, USA)
Leveraging Large Language Models (LLMs) for code generation has increasingly emerged as a common practice in the domain of software engineering. Relevant benchmarks have been established to evaluate the code generation capabilities of LLMs. However, existing benchmarks focus primarily on sequential code, lacking the ability to effectively evaluate LLMs on concurrent code generation. Compared to sequential code, concurrent code exhibits greater complexity and possesses unique types of bugs, such as deadlocks and race conditions, that do not occur in sequential code. Therefore, a benchmark for evaluating sequential code generation cannot be useful for evaluating concurrent code generation with LLMs. To address this gap, we designed a benchmark CONCUR specifically aimed at evaluating the capability of LLMs to generate concurrent code. CONCUR consists of a base set of 43 concurrency problems derived from a standard concurrency textbook, together with 72 validated mutant variants, resulting in 115 total problems. The base problems serve as the semantic core of the benchmark, while the mutants expand linguistic and structural diversity. We conducted an evaluation of a range of LLMs on CONCUR, highlighting limitations of current models. Overall, our work provides a novel direction for evaluating the capability of LLMs to generate code with focus on concurrency.
Article Search Artifacts Available Article: issta26main-p2402-p (type: Full Paper) doi:10.1145/3832286

CONCUR: Benchmarking LLMs for Concurrent Code Generation — Artifact (doi:10.5281/zenodo.20695668): CONCUR is a benchmark for evaluating LLMs on concurrent Java code generation. It consists of 115 concurrency problems, an automated evaluation pipeline using Java PathFinder (JPF) for model checking, and results from 23 LLMs. The artifact includes: | Component | Description | |---|---| | `prompts.csv` | 115 ...

PatchPorter: LLM-Driven Security Patch Porting via Version Tracing and Context Selection for NPM
Zeliang Yu, Ming Wen, Zichao Wei, Yulun Wu, Deqing Zou, and Hai Jin
(Huazhong University of Science and Technology, China; Chinese University of Hong Kong, China)
Third-Party Libraries are widely used in modern software development, yet their vulnerabilities pose serious security risks. This issue is particularly severe in the NPM ecosystem, where high-risk 1-day vulnerabilities can remain unpatched for extended periods. Although upgrading to the latest patched version is commonly recommended, it often causes major compatibility issues. Patch porting offers an effective solution to this challenge. However, existing patch porting methods are mainly designed for C and have two limitations when applied to NPM. First, they can be brittle in precisely localizing fix locations when substantial semantic gaps exist across versions. Second, they rely on either excessive or insufficient context for patch adaptation, which may introduce redundant information and increase the risk of hallucination. This paper introduces PatchPorter, a method for single-branch security patch porting in NPM that builds on Large Language Models (LLMs). PatchPorter addresses these limitations through two modules. The localization module leverages LLM semantic understanding and code evolution analysis over version histories maintained by version control systems to identify fix locations. The context selection module analyzes patch dependencies to select minimal yet sufficient context around the localized fix location. The selected context is used as input for the LLM to generate the patch. We construct a dataset of 112 NPM vulnerabilities with Proof- of-Concepts for dynamic validation. Experimental results show that PatchPorter significantly surpasses other methods in accuracy, achieving a 26.23% improvement over the best-performing baseline and a 70.59% increase on the most difficult tasks. Its ability to handle various vulnerability types highlights its practical value. Additional results confirm that both of its main modules also outperform alternative approaches.
Article Search Artifacts Available Article: issta26main-p2488-p (type: Full Paper) doi:10.1145/3832287

PatchPorter: LLM-Driven Security Patch Porting via Version Tracing and Context Selection for NPM (doi:10.5281/zenodo.21959756): This is the research artifact accompanying the paper “PatchPorter: LLM-Driven Security Patch Porting via Version Tracing and Context Selection for NPM.” It provides the implementation of an automated pipeline for tracing security fixes across software versions, locating vulnerable code, selecting repair context, ...

AsyncLeakBench: A Curated Benchmark of Asynchronous Resource Leaks in Open-Source Java Projects
Jinyoung Kim, Jinseok Heo, Dongwook Choi, and Eunseok Lee
(Sungkyunkwan University, Republic of Korea)
Asynchronous programming is widely used in modern Java software, including server-side processing, network I/O, reactive streams, task scheduling, and RPC communication. Unlike synchronous executions, resource creation and release in asynchronous programs are distributed across callbacks, Future/Promise chains, threads, and scheduler boundaries. Consequently, resource lifecycles may depend on execution ordering, races, cancellation, and timeouts, making resource leaks a significant source of performance degradation and reliability failures. Prior research and public datasets have largely focused on synchronous resource leaks, which typically involve missing close calls or unhandled exceptional paths within a single call stack. In asynchronous environments, however, resource release may depend on callback execution, Future completion, and scheduler decisions, and the release path may change under cancellation, timeouts, or reordered execution. Existing datasets therefore do not adequately capture the triggering conditions and repair strategies of asynchronous resource leaks, limiting the systematic evaluation of detection and automated repair techniques. To address this gap, we present AsyncLeakBench, a public benchmark of real-world asynchronous resource leak defects fixed in open-source Java projects. Using a semi-automatic mining workflow, we collected 16,242 candidate defect–patch pairs from 31 open-source Java projects. Through iterative filtering, duplicate removal, async-specific validation, and manual inspection of resource lifecycles and patches, we identified 902 high-confidence defect–patch pairs. We further classify these cases into 11 categories that characterize major triggers and repair strategies involving cancellation, timeouts, scheduler boundaries, and other asynchronous events. An initial evaluation of existing resource leak detectors reveals limited effectiveness, particularly for leaks triggered by cancellation and timeouts. AsyncLeakBench provides a realistic and reproducible basis for evaluating resource leak detection and repair, static and dynamic analysis, fault localization, and LLM-based debugging. By characterizing asynchronous resource leaks as a distinct defect class and providing a standardized benchmark, this work enables systematic research on their detection, localization, and repair.
Article Search Article: issta26main-p2489-p (type: Full Paper) doi:10.1145/3832288

Systematically Cover SQL Syntactic Structures via 𝑘-Sequence
Hongtao Zhou, Yingying Zheng, Yu Gao, Jiansen Song, Xudong Xie, Rui Yang, Ziyu Cui, Wensheng Dou, and Jun Wei
(Institute of Software Chinese Academy of Sciences, China)
Testing Relational Database Management Systems (RDBMSs) is inherently challenging because SQL, the primary language for interacting with RDBMSs, exhibits a vast and highly complex grammar with hundreds of interdependent production rules in the Extended Backus--Naur Form. While existing grammar-based testing techniques have made progress in covering SQL syntactic structures, they predominantly focus on parent-child relationships in derivation paths, which capture vertical expansions from a non-terminal to its alternatives. However, they overlook an equally critical dimension, sibling-like relationships, which capture co-occurring alternatives across derivation paths. This oversight results in insufficient coverage of intricate syntactic interactions that may trigger unique behaviors or latent bugs in RDBMSs.
In this work, we propose k-sequence, a novel coverage criterion that characterizes syntactic structures as ordered sequences of k alternatives encountered during derivation. By simultaneously capturing both vertical parent-child and horizontal sibling-like relationships in the SQL syntactic structures, k-sequence provides a unified framework for comprehensive SQL syntactic coverage. Based on this criterion, we develop KSeqFuzz, a directed fuzzing approach that systematically generates SQL statements to explore previously unseen k-sequences, achieving deeper and broader testing coverage. We implement and evaluate KSeqFuzz on four widely-deployed RDBMSs, i.e., MySQL, MariaDB, TiDB, and OceanBase. In total, KSeqFuzz detects 58 new unique bugs, including 6 critical crashes. Evaluation results demonstrate that KSeqFuzz outperforms state-of-the-art baselines, detecting 26% more unique bugs during 24-hour testing campaigns.
Article Search Article: issta26main-p2496-p (type: Full Paper) doi:10.1145/3832289

IssueExec: A Test-Driven Approach for Localizing Software Engineering Issues
Jiawei Liu, Yun Lin, Chenyan Liu, Yu Qian, Yiming Liu, Jiaxin Chang, Weinan Zhang, and Linpeng Huang
(Shanghai Jiao Tong University, China; Shanghai Innovation Institute, China; National University of Singapore, Singapore)
Issue localization, which identifies code locations requiring modification from issue descriptions, is a critical step in automated software maintenance. Existing approaches predominantly attempt to directly align issue descriptions with code elements, yet often struggle due to the inherent abstraction gap between the issue description and code implementation. Seeking alternative signals, our theoretical analysis suggests that test suites can serve as executable proxies for requirements, reducing localization uncertainty by 7.73 bits of entropy on average. A large-scale empirical study on 18 repositories validates this premise: existing tests cover 96.98% of ground-truth files, and the two-hop pathway yields stronger semantic connectivity than direct matching in 82.4% of cases. Despite their potential, leveraging tests for localization faces two key challenges: the semantic gap separating issue descriptions from test identifiers, and the substantial noise in execution traces from infrastructure code. To address these, we propose IssueExec, which bridges the semantic gap through domain-knowledge-enhanced test representations and filters noise via hierarchical trace analysis. Experiments on SWE-bench Lite show that IssueExec achieves state-of-the-art performance, improving function-level Recall@1 by 41.57% over the strongest baseline. When integrated into the Agentless pipeline, IssueExec resolves 17.72% more issues, demonstrating practical downstream benefits.
Article Search Article: issta26main-p2497-p (type: Full Paper) doi:10.1145/3832290

Mathematically-Guided Detection of Floating-Point Errors
Youshuai Tan, Zhanwei Zhang, Haonan Zhang, Lianyu Zheng, Zishuo Ding, Jinfu Chen, and Weiyi Shang
(Hong Kong University of Science and Technology (Guangzhou), China; University of Waterloo, Canada; Wuhan University, China)
Floating-point computations are important for modern scientific and engineering software, especially for safety-critical systems, yet only a small subset of inputs typically trigger substantial numerical errors. Detecting such error-inducing inputs and the underlying bugs is therefore essential for improving their security and reliability. Existing techniques commonly rely on either oracle-driven exploration that repeatedly compares against high-precision references or search-driven heuristics. Despite the improvements made, they remain limited by (1) Expensive computation of high-precision oracles and (2) Lack of long-range convergence, which often requires dense probing near narrow error-inducing regions and expensive computation.
We propose MGDE (Mathematically-Guided Detection of floating-point Errors), a method that replaces trial-and-error exploration with mathematically defined targets and directed convergence. MGDE first uses condition-number theory to identify numerically unstable atomic operations without invoking expensive high-precision oracles during exploration. MGDE exploits the observation that extreme condition numbers occur near structured boundaries (e.g., cancellation points and singularities), reformulating detection as a numerical root-finding problem. By solving the resulting objectives with the Newton–Raphson method, MGDE can steer inputs toward error-prone regions from far-away initializations.
We evaluate MGDE on GNU Scientific Library (GSL) functions and compare against two state-of-the-art baselines, ATOMU and FPCC, using triggered bugs as the primary metric. On 88 single-input functions, MGDE triggers 80 numerically validated bugs across 47 functions, outperforming ATOMU (70 bugs in 46 functions) and FPCC (53 bugs in 42 functions). MGDE is also faster: ATOMU and FPCC require 42.71× and 11.17× the exploration time of MGDE, respectively. Regarding multi-input functions, we evaluate MGDE under two complementary settings. On the native multi-input dataset of FPCC, MGDE detects 28 triggered bugs, while FPCC finds 23 bugs. MGDE also takes 8.91 seconds in total, compared with 2,100 seconds used by FPCC. On an additional external benchmark of 18 dual-input GSL functions, MGDE detects nine bugs not found by FPCC. Overall, MGDE substantially advances the state-of-the-art in both effectiveness and efficiency, and we report 16 previously unknown GSL bugs, which have been confirmed by the GSL community.
Article Search Article: issta26main-p2500-p (type: Full Paper) doi:10.1145/3832291

TracePilot: Self-Verifiable Framework for Decentralized Applications Fault Localization across Transactions
Xuanyu Zhu, Zhiying Wu, Tao Wang, Ying Yan, Wei Zhou, Jiajing Wu, Zigui Jiang, and Zibin Zheng
(Sun Yat-sen University, China; Guangdong Engineering Technology Research Center of Blockchain, Zhuhai, China; Ant Digital Technologies, Hangzhou, China)
Decentralized Applications (DApps) serve as a critical technical underpinning for business logic and user interaction within the blockchain-powered Web3 ecosystem. However, DApps are prone to faults, and localizing these faults within their intricate and often interconnected logic is a particularly time-consuming process, frequently taking tens of hours and leading to substantial economic losses for developers. Existing state-of-the-art DApp fault localization methods, e.g., FaultSeeker, cannot capture cross-transaction fault logic and produce verifiable diagnostic reports. Therefore, security experts have to spend substantial time manually verifying results and devising fixes.
In this paper, we present TracePilot, a large language model (LLM)-based framework that automates DApp fault localization in two phases: distilling global fault insights from transaction sequences and then performing focused trace exploration to isolate the faulty logic. Crucially, we propose a patch verification mechanism that treats attack-blocking patches as executable evidence for fault localization while flagging potential overfitting risks for expert review. This mechanism improves result trustworthiness and reduces manual verification costs.
Evaluated on a dataset of 149 real-world cases, TracePilot achieves a 71.14% Top-1 Recall. In the single-transaction fair comparison, it achieves 72.73%, substantially outperforming the state-of-the-art method at 32.23%. On cross-transaction cases, TracePilot achieves a 64.29% Top-1 Recall. The proposed algorithm is being integrated into the contract security agent developed by Ant Digital Technologies. Moreover, to facilitate further research, our code and dataset are publicly available online: https://github.com/feiqiuaaaa/TracePilot.
Article Search Article: issta26main-p2516-p (type: Full Paper) doi:10.1145/3832292

Empowering Lightweight Language Models for Security Code Review via Context-Aware Distillation
Zixiao Zhao, Yanjie Jiang, Hui Liu, and Lu Zhang
(Peking University, China; Tianjin University, China; Beijing Institute of Technology, China)
Security code review, which specifically examines software from a security perspective, is indispensable for preempting vulnerabilities and bolstering software reliability. However, existing automated approaches face a dilemma. They either rely on large language models with prohibitive deployment costs, or employ lightweight models that struggle with complex reasoning and global contextual understanding. In this paper, we propose LSCR, a context-aware distillation approach that empowers lightweight language models for security code review by distilling static-analysis–style security rationale from powerful teacher models. Rather than directly transferring final review outputs, LSCR guides student models to internalize how high-level security judgments are systematically derived from low-level code evidence, leveraging repository-level context during training. By embedding this evidence-driven analysis paradigm into lightweight models, LSCR enables more reliable security code reviews under constrained model capacity. When compared with competitive lightweight baselines, LSCR attains an average improvement of over 12% in security issue classification accuracy and more than 13.3% gains in BLEU score for review generation. Human evaluation reveals that LSCR increases the proportion of instrumental reviews by 52.6% on average over state-of-the-art baselines, while reducing misleading feedback by 22.6%. LSCR effectively narrows the performance gap between small-scale and large-scale models, making on-premise security code review more feasible in practice.
Article Search Article: issta26main-p2517-p (type: Full Paper) doi:10.1145/3832293

EvidenT: An Evidence-Preserving Framework for Iterative System-Level Package Repair
Chenyu Zhao, Minghua Ma, Shenglin Zhang, Zeshun Huang, Yongqian Sun, Chetan Bansal, Saravan Rajmohan, and Dan Pei
(Nankai University, China; Microsoft, USA; Tsinghua University, China)
Frequent toolchain updates and the expanding diversity of instruction set architectures (ISAs) have made large-scale system-level software package repair a critical task. Diagnosing and repairing build failures remains challenging due to heterogeneous failure evidence, complex dependency constraints, and architecture-specific build conventions. While recent LLM-based repair methods have shown promise for project-level source code fixes, they struggle with system-level repair where failures involve multi-language artifacts (e.g., build recipes, scripts, and source archives) and require iterative validation through external build services. In this paper, we first conduct a systematic empirical study of real-world system-level build failures. Our findings reveal that 72% of successful repairs primarily involve adjustments to build configurations, dependencies, or environment settings rather than isolated source-code modifications, suggesting that effective repair must prioritize packaging logic and iterative feedback. Motivated by these insights, we propose EvidenT, an evidence-preserving repair framework that decouples iteration-aware evidence management from tool execution. EvidenT comprises (1) an external Build Service for reproducible build execution and feedback; (2) an Evidence-Preserving Repair Controller that performs cross-modal fusion of repair history, knowledge context, and build artifacts; and (3) an automated Repair Orchestrator that executes a suite of modular tools for failure localization and system-level repair actions within a closed-loop validation environment. We evaluate EvidenT on a benchmark of 219 real-world RISC-V package build failures. EvidenT successfully repairs 118 packages (53.88%), substantially outperforming state-of-the-art agentic baselines (20.55%) and direct LLM-based repair (1.83%). To demonstrate its architectural generality, we extend EvidenT to other ISAs by updating only ISA-specific knowledge context. In preliminary experiments, it achieves success rates of 41.77% on aarch64 and 46.99% on x86_64, showcasing its robustness across diverse hardware ecosystems.
Article Search Artifacts Available Article: issta26main-p2564-p (type: Full Paper) doi:10.1145/3832294

EvidenT ISSTA 2026 Artifact (doi:10.5281/zenodo.20972389): This artifact accompanies the paper “EvidenT: An Evidence-Preserving Framework for Iterative System-Level Package Repair”. It contains the EvidenT implementation, configurations and scripts for smoke testing, Docker- and OBS-based validation, and end-to-end and batch repair runs, together with documentation for ...

CARE: Cascading Impact-Aware Compliance Test Suite Evolution under Regulatory Changes
Zhiyi Xue, Xiaohong Chen, and Min Zhang
(East China Normal University, China)
In response to frequent changes in regulatory rules, this paper proposes CARE, a cascading impact-aware framework for automated compliance testing evolution. Existing approaches often suffer from over-reuse or missed updates because they treat rule changes in isolation and ignore complex inter-dependencies across testing artifacts. This paper highlights cascading impact propagation as a central challenge in regulation-driven test maintenance and shows that shifting from isolated rule handling to cascading impact-aware evolution is essential for achieving both high test quality and maintenance efficiency. Specifically, our CARE framework addresses this challenge by constructing a unified four-layer cascading relation model spanning Rule-Requirement-Scenario-Test Case, enabling fine-grained traceability across abstraction levels. By explicitly modeling how rule changes propagate and amplify along this chain, the framework precisely identifies impacted scenarios and test cases that need updating, while safely maximizing the reuse of unaffected ones. Experiments conducted on real-world compliance testing tasks across multiple domains show that CARE achieves an average F1 of 90.3% on updated test suites, outperforming existing methods by up to 164% and approaching expert-level effectiveness. Ablation studies further demonstrate that explicit cascading impact modeling and handling are key contributors to these improvements. In addition, CARE substantially reduces manual effort and improves test maintenance efficiency, and indicates strong cross-domain generalization.
Article Search Article: issta26main-p2703-p (type: Full Paper) doi:10.1145/3832295

Fine-Grained Privacy Leakage Detection in OpenHarmony Apps
Aohan Mei, Guangliang Yang, Xinming Guo, Yi Wang, Fuan Gui, and Min Yang
(Fudan University, China)
In recent years, the distributed operating system OpenHarmony has gained significant popularity. As the number of OpenHarmony apps grows rapidly, privacy abuse and data leakage have emerged as critical concerns. However, the untyped and highly flexible nature of Ark bytecode poses substantial challenges. We propose HScope, a novel fine-grained program analysis framework designed to directly analyze OpenHarmony app bytecode and identify privacy risks. HScope employs abstract interpretation to model the dynamic behaviors of OpenHarmony apps, enabling precise resolution of indirect calls and the complex inter-component communication mechanisms. We evaluate HScope on a dataset of 300 real-world OpenHarmony apps. The results demonstrate that HScope is both effective and comprehensive, successfully uncovering 39 previously unknown privacy issues (corresponding to 27 apps). These findings highlight HScope’s potential as a practical and scalable solution for securing the evolving OpenHarmony ecosystem.
Article Search Artifacts Available Article: issta26main-p2727-p (type: Full Paper) doi:10.1145/3832296

Reproduction package for article 'Fine-Grained Privacy Leakage Detection in OpenHarmony Apps' (doi:10.5281/zenodo.21507308): This artifact contains the source code of HScope, the static analysis framework introduced in the paper, together with OH-Bench, a benchmark suite for evaluating privacy-leak detection in OpenHarmony applications. HScope analyzes Ark bytecode, reconstructs an intermediate representation, performs semantic and taint ...

An Empirical Study and Benchmark of Kubernetes Misconfiguration Scanners
Haeun Eom, Bohyun Suk, and Sungjae Hwang
(Sungkyunkwan University, Republic of Korea; Republic of Korea Army, Republic of Korea)
Kubernetes is a widely adopted container orchestration framework, yet misconfigurations remain a leading cause of cloud security incidents and a major challenge for practitioners. Automated security scanners are commonly used to detect such misconfigurations, but their effectiveness has not been systematically evaluated. As a result, it remains unclear which tools can be trusted, what misconfigurations they reliably detect, and to what extent they improve Kubernetes security.
This paper presents the first systematic investigation of ten prominent Kubernetes security scanners that are actively used in practice. We begin by examining the misconfigurations that each scanner claims to detect. Although scanners provide documentation of their coverage, these descriptions are written in natural language and are often ambiguous, making it unclear what is actually detected. To address this issue, we manually analyze scanner implementations to identify their precise detection targets and validate them through dynamic testing. Our analysis reveals that scanners frequently adopt different criteria for the same documented misconfiguration, exposing significant inconsistencies caused by ambiguous specifications. Building on these results, we introduce the first comprehensive benchmark for Kubernetes misconfigurations, covering all misconfigurations targeted by the ten scanners. The benchmark includes 4,109 misconfiguration files for static and dynamic analysis and 144 shell scripts for dynamic analysis, encompassing 281 unique misconfigurations. Using this benchmark, we conducted an empirical evaluation of ten scanners. Our results show that, in static scanning, Kubescape achieves the highest recall (43.8%), and it also performs best in dynamic analysis with an recall of 62.3%. We further analyze the strengths and limitations of each scanner, identifying coverage gaps that significantly affect detection effectiveness.
Our findings provide practical guidance for practitioners selecting Kubernetes security scanners and highlight key challenges that should be addressed by the software engineering community. Moreover, the proposed benchmark establishes a foundation for future research on Kubernetes security.
Article Search Artifacts Available Article: issta26main-p2747-p (type: Full Paper) doi:10.1145/3832297

An Empirical Study and Benchmark of Kubernetes Misconfiguration Scanners (doi:10.5281/zenodo.21288745): This artifact provides a Kubernetes misconfiguration benchmark designed to evaluate the full range of policies checked by the selected scanners. It was manually constructed based on an analysis of the scanners’ policy implementation code and the security frameworks underlying their policies. Each benchmark file ...

Towards Iterative End-to-End Software Development: A Feature-Driven Multi-agent Framework
Junwei Liu, Chen Xu, Chong Wang, Tong Bai, Weitong Chen, Kaseng Wong, Yiling Lou, and Xin Peng
(Fudan University, China; Nanyang Technological University, Singapore; University of Illinois at Urbana-Champaign, USA)
Recent advances in large language model agents offer the promise of automating end-to-end software development from natural language requirements. However, existing approaches largely adopt linear, waterfall-style pipelines, which oversimplify the iterative nature of real-world development and struggle with complex, larger-scale projects. To address these limitations, we propose EvoDev, an iterative software development framework inspired by feature-driven development. EvoDev decomposes user requirements into a set of user-valued features and constructs a Feature Map, a directed acyclic graph that explicitly models dependencies between features. Each feature node in the feature map maintains multi-layer contexts, including business logic, software design, and code implementation, which are propagated along dependencies to provide context for subsequent development iterations. We evaluate EvoDev on challenging Android development tasks and show that it improves Function Completeness by 57.3% over the best-performing baseline, Claude Code, while achieving 16.0%–58.5% improvements over single-agent baselines with different base LLMs. These results highlight the importance of feature decomposition, dependency modeling, context propagation, and workflow-aware agent design for end-to-end software development. Moreover, our work summarizes practical insights for designing iterative, LLM-driven development frameworks and informs future training of base LLMs to better support iterative software development.
Article Search Article: issta26main-p2799-p (type: Full Paper) doi:10.1145/3832298

Silence of Commit Messages: An Empirical Study for Vulnerability Commit Message Generation using Large Language Models
Hao Shen, Ming Hu, Jiaye Li, Xiaofei Xie, and Mingsong Chen
(East China Normal University, China; Singapore Management University, Singapore)
The vulnerability commit message serves as crucial metadata for maintaining software within version control systems. Nonetheless, manually crafted vulnerability commit messages often lack detail or exhibit inconsistent formatting. Recently, the growing use of Large Language Models (LLMs) for code and natural language comprehension has opened avenues to automate the crafting of these messages. This paper systematically and thoroughly explores the generation of security patch commit messages in the context of LLMs, delving into topics such as dataset construction, evaluation method design, and the relationship between vulnerability types and submission structure. First, we explore the elements of commit messages using LLMs and integrate a questionnaire survey to pinpoint four essential types of information: summary, background, impact, and fix, which are essential for developers. This aims to establish a structured dataset of bug submissions and assess its quality. Next, we examine general automated evaluation techniques for assessing LLM-generated commit messages and find that GPT-3.5's evaluation methods align more closely with human judgment. Then, we conduct an organized investigation into how LLM generation effects vary across three principal vulnerability types, uncovering that LLMs' adaptability differs across vulnerabilities. Furthermore, we perform an exhaustive examination of generation quality across various components and find that LLMs excel at generating summaries but struggle to produce impact details. In particular, the smallest DeepSeek-Coder shows a semantic retention advantage in crafting backgrounds, whereas DeepSeek-V3 struggles with impact aspects. Lastly, we investigate the effects of different prompting strategies (e.g., zero-shot, few-shot prompts) and parameter settings (e.g., temperature and top_p) on the quality of commit message generation, finding that prompt and parameter configurations critically influence output quality, with model sensitivity varying.
Article Search Article: issta26main-p2928-p (type: Full Paper) doi:10.1145/3832299

Characterizing and Repairing Obsolete Android GUI Tests under UI Evolution
Shiwen Song, Yiheng Xiong, Wenbo Guo, Manqi Sun, Jiaolong Kong, and Xiaofei Xie
(Singapore Management University, Singapore; Nanyang Technological University, Singapore; University of Hong Kong, China)
Graphical user interface (GUI) tests are widely used in regression testing of mobile applications (apps) to validate app behavior from the user's perspective. However, frequent app evolution, such as UI redesigns and feature updates, often renders existing GUI tests obsolete, even when underlying functionality remains unchanged. Automatically repairing such tests is critical for maintaining test suites and reducing substantial manual effort. Despite the practical importance, there is still a lack of a systematic understanding of the characteristics of obsolete GUI tests and a publicly available benchmark to support their study. To fill this gap, we construct a benchmark comprising 736 obsolete GUI tests collected from 36 real-world mobile apps across 668 historical versions. We then conduct a large-scale empirical study that reveals two major challenges in repairing obsolete GUI tests: First, identifying the intended target widget is difficult because widget attributes are frequently missing or unstable, and visually similar widgets may correspond to different functionalities. Second, the target widget is often no longer directly reachable from the failure state, as it may be hidden behind additional UI interactions or relocated to another page. To address these challenges, we further propose GUIRevive, an automated GUI test repair approach that addresses semantic ambiguity in widget identification and target unreachability under UI evolution by integrating semantic-aware reasoning, functionality-preserving validation, and goal-guided UI exploration. Our evaluation shows that GUIRevive successfully repairs 86.4% of obsolete GUI tests and significantly outperforms state-of-the-art repair techniques by up to 220%. Moreover, GUIRevive has been deployed in industrial settings, achieving a 93% repair success rate on industrial mobile apps.
Article Search Artifacts Available Article: issta26main-p2990-p (type: Full Paper) doi:10.1145/3832300

Characterizing and Repairing Obsolete Android GUI Tests under UI Evolution (doi:10.5281/zenodo.21699913): This software artifact contains the source code of GUIRevive, accompanying the paper "Characterizing and Repairing Obsolete Android GUI Tests under UI Evolution". The artifact includes the implementation of GUIRevive, configuration files, experimental scripts, and documentation required to reproduce the experiments ...

Project-Scale Statement-Level Fault Localization via Multi-view Semantic Learning and Pairwise Reranking
Hongwei Yu, Xu Wang, Jian Zhang, Xiangxin Meng, Jiarui Li, Yang Liu, and Chunming Hu
(Beihang University, China; Nanyang Technological University, Singapore)
Statement-level fault localization (FL) is critical for effective software debugging, as it enables developers to precisely identify faulty lines of code. While traditional spectrum-based, mutation-based, and deep learning-based FL techniques have achieved notable progress, they remain limited in modeling rich fault semantics. Recent advances in large language models (LLMs) offer new opportunities for FL due to their strong capacity for semantic understanding and reasoning over bugs. However, existing LLM-based FL approaches largely treat fault localization as isolated, code-centric prediction, limiting their ability to perform the holistic fault reasoning required for precise statement-level localization.
In this paper, we propose FaultScape, a novel LLM-based framework for project-scale, statement-level FL that formulates FL as a multi-view semantic learning and reasoning problem. FaultScape addresses the limitations of existing approaches through three key components. First, we introduce a joint contrastive fine-tuning strategy that trains LLMs on large-scale bug-fix data to explicitly learn fault semantics from multiple complementary views. View-specific fault semantics, including code semantics, fault type, root cause, and repair intent, are learned via supervised binary classification, while cross-view semantic consistency is enforced through contrastive learning. These two objectives are jointly optimized within a unified training loss. The fine-tuned models extract multi-view fault likelihoods as semantic features for each suspicious statement. Second, we adopt a dynamic feature integration module that combines these semantic features with spectrum-based and mutation-based execution features, producing an initial suspiciousness statement ranking. Third, we design an LLM-based, test-guided pairwise re-ranking strategy that explicitly compares highly suspicious candidate statements using failing-test context. By estimating relative fault likelihoods through pairwise comparison rather than independent scoring, the model produces a refined statement-level ranking. We evaluate FaultScape on Defects4J v1.2.0, where it localizes 112/171/196 bugs at Top-1/3/5 out of 395, outperforming state-of-the-art DL-based and LLM-based baselines. On leakage-free benchmarks, FaultScape further localizes 11/18/20 bugs at Top-1/3/5 on ConDefects (31 bugs) and 12/21/22 bugs on GHRB (34 bugs), demonstrating strong generalization to unseen projects. These results show that combining multi-view fault semantics with contrastive, failure-guided reasoning substantially improves the effectiveness and robustness of statement-level fault localization.Statement-level fault localization (FL) is critical for effective software debugging, as it enables developers to precisely identify faulty lines of code. While traditional spectrum-based, mutation-based, and deep learning-based FL techniques have achieved notable progress, they remain limited in modeling rich fault semantics. Recent advances in large language models (LLMs) offer new opportunities for FL due to their strong capacity for semantic understanding and reasoning over bugs. However, existing LLM-based FL approaches largely treat fault localization as isolated, code-centric prediction, limiting their ability to perform the holistic fault reasoning required for precise statement-level localization.
In this paper, we propose FaultScape, a novel LLM-based framework for project-scale, statement-level FL that formulates FL as a multi-view semantic learning and reasoning problem. FaultScape addresses the limitations of existing approaches through three key components. First, we introduce a joint contrastive fine-tuning strategy that trains LLMs on large-scale bug-fix data to explicitly learn fault semantics from multiple complementary views. View-specific fault semantics, including code semantics, fault type, root cause, and repair intent, are learned via supervised binary classification, while cross-view semantic consistency is enforced through contrastive learning. These two objectives are jointly optimized within a unified training loss. The fine-tuned models extract multi-view fault likelihoods as semantic features for each suspicious statement. Second, we adopt a dynamic feature integration module that combines these semantic features with spectrum-based and mutation-based execution features, producing an initial suspiciousness statement ranking. Third, we design an LLM-based, test-guided pairwise re-ranking strategy that explicitly compares highly suspicious candidate statements using failing-test context. By estimating relative fault likelihoods through pairwise comparison rather than independent scoring, the model produces a refined statement-level ranking. We evaluate FaultScape on Defects4J v1.2.0, where it localizes 112/171/196 bugs at Top-1/3/5 out of 395, outperforming state-of-the-art DL-based and LLM-based baselines. On leakage-free benchmarks, FaultScape further localizes 11/18/20 bugs at Top-1/3/5 on ConDefects (31 bugs) and 12/21/22 bugs on GHRB (34 bugs), demonstrating strong generalization to unseen projects. These results show that combining multi-view fault semantics with contrastive, failure-guided reasoning substantially improves the effectiveness and robustness of statement-level fault localization.
Article Search Article: issta26main-p3023-p (type: Full Paper) doi:10.1145/3832301