AgenticDev '26: "Context Inheritance: A Git-Native ..."
Context Inheritance: A Git-Native Architecture and Pre-registered Study of Repository Context for AI Coding Agents
Pavel Kerbel, Milana Kerbel, Vitali Abramov, and Liat Abramov
(Metatron Research, Israel)
AI coding agents repeatedly rediscover project constraints, because repositories version code but not operating knowledge: the decisions taken, the alternatives rejected, the constraints found during execution. We introduce the Repository Context Layer (RCL) — the repository carries its own agent-facing context, versioned with the code, deterministically discoverable, consulted by contract, and writable by agents only under human review, in a minimal default format, context.md — and study, in a pre-registered experiment on SWE-bench Verified, when it helps. The organizing observation is a capability-dependent author–reader pattern we call context inheritance: repository context behaves like senior knowledge, and across the levels we observe its value flows toward weaker readers. A small local executor gains a large navigational benefit from context authored by a more capable model (+26 pp localization on topic-matched tasks, $p < 0.0001$) but nothing from context it wrote itself (+0.0 pp); with two author levels drawn from different model families, we report this as a pattern consistent with a capability gradient rather than an isolated causal variable. At the frontier the axes shift: an agent appears to gain from its own failure-derived lessons, but not from answer-derived ones (+14.6 pp resolve on constraint-sharing tasks; $p = 0.041$ uncorrected, $0.081$ after Holm correction within its registered family, so we report the direction rather than a confirmed effect; −32% tokens per resolved task); how context is delivered matters (exploratorily, a sharded store the agent reads selectively outperformed a monolith at a third of the tokens at the local tier); and on a broad random sample a frontier agent sits at a resolve ceiling that no context improves. The design consequence is concrete: version repository context as a first-class artifact, author it with the strongest writer available, and expect the benefit in cheaper agents that read it.
Article Search
Article: asews26agenticdevmain-p22-p (type: Full Paper) doi:10.1145/3843282.3844425
AgenticDev '26: "Context Inheritance: A Git-Native ..."
Context Inheritance: A Git-Native Architecture and Pre-registered Study of Repository Context for AI Coding Agents
Pavel Kerbel, Milana Kerbel, Vitali Abramov, and Liat Abramov
(Metatron Research, Israel)
AI coding agents repeatedly rediscover project constraints, because repositories version code but not operating knowledge: the decisions taken, the alternatives rejected, the constraints found during execution. We introduce the Repository Context Layer (RCL) — the repository carries its own agent-facing context, versioned with the code, deterministically discoverable, consulted by contract, and writable by agents only under human review, in a minimal default format, context.md — and study, in a pre-registered experiment on SWE-bench Verified, when it helps. The organizing observation is a capability-dependent author–reader pattern we call context inheritance: repository context behaves like senior knowledge, and across the levels we observe its value flows toward weaker readers. A small local executor gains a large navigational benefit from context authored by a more capable model (+26 pp localization on topic-matched tasks, $p < 0.0001$) but nothing from context it wrote itself (+0.0 pp); with two author levels drawn from different model families, we report this as a pattern consistent with a capability gradient rather than an isolated causal variable. At the frontier the axes shift: an agent appears to gain from its own failure-derived lessons, but not from answer-derived ones (+14.6 pp resolve on constraint-sharing tasks; $p = 0.041$ uncorrected, $0.081$ after Holm correction within its registered family, so we report the direction rather than a confirmed effect; −32% tokens per resolved task); how context is delivered matters (exploratorily, a sharded store the agent reads selectively outperformed a monolith at a third of the tokens at the local tier); and on a broad random sample a frontier agent sits at a resolve ceiling that no context improves. The design consequence is concrete: version repository context as a first-class artifact, author it with the strongest writer available, and expect the benefit in cheaper agents that read it.
Article Search
Article: asews26agenticdevmain-p22-p (type: Full Paper) doi:10.1145/3843282.3844425
AgenticDev '26: "Defining and Deploying Agentic ..."
Defining and Deploying Agentic Swarms with the BESSER Low-Code Platform
Adem Ait, Javier Luis Cánovas Izquierdo, and Jordi Cabot
(University of Luxembourg, Luxembourg; Open University of Catalonia, Spain; Luxembourg Institute of Science and Technology, Luxembourg)
Complex software engineering tasks increasingly rely on agentic swarms, i.e., multi-agent systems in which specialized agents reason, plan, and act collaboratively to decompose a problem, propose solutions, review each other’s work, and converge on a decision. Today, such swarms are typically built directly in framework code, or with no-code tools that capture only the workflow, leaving per-agent capabilities and confidence, decision policies at merge points, and deployment without explicit, first-class abstractions. We argue that engineering agentic swarms calls for model-driven abstractions that span four complementary levels of granularity: the whole swarm, the process within a swarm, the merge point within a process, and the individual agent. In this demo we present an end-to-end approach on top of the open-source BESSER low-code platform that realizes these levels through four cross-referenced models, i.e., an agentic BPMN process, a state-machine-based Agent diagram, and UML Component and Deployment diagrams, authored in a web-based modeling editor. From the Deployment diagram, the fully specified swarm is generated and deployed to a Docker engine. We demonstrate the approach with a bug-fixing swarm composed of an Analyzer, two Solvers, and a Reviewer.
Article Search
Article: asews26agenticdevmain-p102-p (type: Short Paper) doi:10.1145/3843282.3844437
AgenticDev '26: "Cost-Effective Repository ..."
Cost-Effective Repository Exploration for Agentic Issue Localization
Mohammad Nour Al Awad and Sergey Ivanov
(ITMO University, Russian Federation)
Repository exploration is a distinct and costly stage of coding-agent pipelines: before generating a patch, an agent must identify which repository files are likely to matter. We study whether this stage can be delegated to lower-cost models while retaining useful localization quality. Using our IssueLoc-Bench, we evaluate five explorer models under the same read-only interactive interface on 499 SWE-bench Verified-derived tasks and 500 tasks from 153 additional repositories. We measure early candidate discovery, top-three gold-file coverage, strict file-set recovery, agent time, and token usage, with paired instance-level uncertainty and repository-clustered sensitivity analysis. The highest-quality explorer leads across localization metrics, but substantially cheaper operating points emerge: depending on the model and evaluation arm, lower-cost explorers retain approximately 78–94% of the reference Hit@3 and 73–92% of its F1 while reducing mean agent time by 41–88% and token usage by 84–95%. The preferred operating point depends on how localization is consumed downstream: ranking and coverage metrics characterize recoverable candidate handoffs, whereas F1 and exact match characterize restrictive file gates. These results support treating repository exploration as an independently measurable and budgetable stage of modular coding agents, with explorer selection guided by the downstream handoff contract.
Article Search
Article: asews26agenticdevmain-p18-p (type: Full Paper) doi:10.1145/3843282.3844422
AgenticDev '26: "Automated Annotation of Test ..."
Automated Annotation of Test Types using LLMs: A Multi-LLM Workflow for Machine Learning Test-Related Commits
Asli Sari, Ayse Tosun, and Gulfem Isiklar Alptekin
(Istanbul Technical University, Türkiye; Galatasaray University, Türkiye)
Test-type annotation classifies commits based on the type of software test introduced or modified. While there are established taxonomies for testing machine learning (ML) software systems, implementing these manually at the commit level can be costly. This makes large language models (LLMs) a promising alternative. Utilizing an eight-category test-type taxonomy, we evaluate 97 manually labeled commits from scikit-learn using GPT-4o, Claude Sonnet, and Qwen 2.5-7B-Instruct-Turbo LLMs through four prompting strategies. This analysis focused on their classification performance, stability, agreement with experts, top-2 candidate recovery, and multi-LLM voting workflows (S1-S4) along with a corresponding analysis on Keras. Notably, GPT-4o in a zero-shot setting demonstrated the highest stability, and permitting a second label led to a reduction in single-prediction errors (e.g., Claude CoT-ZS Hit@1 = [59.79]%, Hit@2 = [77.32]%). Our most effective voting strategy (S1) surpassed the solo baseline, with its superior macro-F1 score (S1 > S2 > S3 > S4) also holding for Keras.
Article Search
Artifacts Available
Article: asews26agenticdevmain-p89-p (type: Extended Abstract (2 pages)) doi:10.1145/3843282.3844433
AgenticDev '26: "Bug Composition: Triggering ..."
Bug Composition: Triggering Multiple Bugs with Agentic Driver Generation
Vishva Arasan, Mahima Chaudhary, Wendy Contreras Martinez, Md Shafiuzzaman, Achintya Desai, and Tevfik Bultan
(University of California at Santa Barbara, USA)
Security vulnerabilities caused by bugs in software do not exist in isolation. Attackers can chain vulnerabilities caused by multiple bugs together to achieve their goals. However, determining whether multiple vulnerabilities can be triggered during a single execution is a difficult problem. In this paper, we present an agentic pipeline for automatically discovering whether a single shared input can trigger multiple bugs. In particular, we introduce an agentic approach that automatically generates predicates and constraint-based harnesses from buggy code, leveraging symbolic execution and SMT solving. We validate our approach using real-world vulnerabilities across three widely used, open source libraries: tcpdump, libxml2, and nDPI. Our pipeline successfully generates concrete shared inputs to trigger multiple bug pairs, including heap buffer overflows in tcpdump, stack and heap memory corruptions in libxml2 and out-of-bounds reads across independent protocol dissectors in nDPI. Furthermore, our methodology correctly identifies and proves the absence of shared inputs when bugs require mutually exclusive constraints. Our work introduces and demonstrates the feasibility of automated bug composition analysis, results of which can be valuable in identifying potential exploit chains and prioritizing critical vulnerabilities for patching.
Article Search
Article: asews26agenticdevmain-p97-p (type: Full Paper) doi:10.1145/3843282.3844434
AgenticDev '26: "MOSAICO: A Multi-agent Infrastructure ..."
MOSAICO: A Multi-agent Infrastructure for Collaborative LLM-Powered Software Engineering
Tuong Manh Vu, Ionut Predoaia, Konstantinos Barmpis, Dimitris Kolovos, and Antonio García-Domínguez
(University of York, UK)
Modern software engineering tasks increasingly call for Artificial Intelligence (AI) assistance that goes beyond a single large language model (LLM) acting alone. We present MOSAICO, an infrastructure for building and orchestrating communities of LLM-powered agents that collaborate to solve software engineering tasks. MOSAICO provides a reference client, a reference agent, and a set of task-specialised agent roles (solution, supervision, and consensus agents) that a collaboration agent invokes according to a workflow to complete a task. We showcase MOSAICO through three concrete demonstrations: a metamodelling use case, where a team of four agents jointly construct and validate a domain metamodel; a UML generation use case, where an agent converts a diagram image into textual notation; and a code documentation use case, where an agent uses client-side tools to generate documentation for existing code. Together, these demonstrations illustrate the generality of MOSAICO: a single, role-based agent infrastructure can support structurally different software engineering tasks with minimal reconfiguration.
Article Search
Article: asews26agenticdevmain-p159-p (type: Short Paper) doi:10.1145/3843282.3844444
AgenticDev '26: "Towards Model-Driven Generation ..."
Towards Model-Driven Generation of Domain-Specific Modelling Agents for Local Language Models
Antonio Garcia-Dominguez, Ionut Predoaia, Tuong Manh Vu, Konstantinos Barmpis, and Dimitris Kolovos
(University of York, UK)
Current frontier LLMs leverage vast training datasets with knowledge about a broad range of programming languages and frameworks, and can often solve tasks with minimal prompting. However, there are scenarios where remote LLMs are unsuitable, due to cost, privacy, security, or intellectual property concerns, and local language models need to be used. These smaller local language models typically require more careful guidance in order to achieve acceptable performance. In this paper, we evaluate the impact of using local models (in the 14-31B range) for object-oriented modelling, and we propose an approach for partially automating the creation of agents that "think in code" to solve such problems, instead of directly producing an output. We compare the direct invocation of Claude Sonnet 4.6 with that of several open-weight models, and with the use of those open-weight models while "thinking in code" as proposed. We observe that the best open-weight models, when "thinking in code", can produce models with similar Jaccard similarity scores to the model solutions in the Golden UML ModelSet dataset as Sonnet, although at the cost of increased token consumption. Based on the observed potential for generating domain-specific agents that reduce the demands on the LLM and therefore on the hardware requirements, we set out several lines of further work.
Article Search
Artifacts Available
Article: asews26agenticdevmain-p58-p (type: Full Paper) doi:10.1145/3843282.3844429
AgenticDev '26: "Documentation Retrieval by ..."
Documentation Retrieval by AI Coding Agents: An Empirical Study of HTTP Behavioral Signatures
Oleksii Borysenko
(Cisco DevNet, Spain)
AI coding agents increasingly retrieve developer documentation on a user's behalf, but conventional portal analytics reveal little about this interaction. We characterize server-side HTTP fingerprints from nine coding agents and six AI assistant services accessing one controlled, public documentation endpoint in three trials each. Within this trial window, the tools exhibited stable, distinguishable request sequences, runtimes, User-Agent strings, and header patterns, usually retrieving the target in one or two requests. The results characterize observable HTTP behavior; they do not measure retrieval quality or downstream task performance.
Article Search
Article: asews26agenticdevmain-p169-p (type: Extended Abstract (2 pages)) doi:10.1145/3843282.3844445
AgenticDev '26: "Execution-First Synthetic ..."
Execution-First Synthetic Tool-Use Trace Generation for LLM Agents
Hafsa Ouajdi, Francesco Giannuzzo, Alaa Boukhary, Paolo Papotti, Gerard Conangla, and Adam Elwood
(EURECOM, France; Aily Labs, Munich, Germany)
Agentic systems increasingly operate through executable workflows: they search artifacts, invoke tools, inspect structured observations, and query databases. Training such agents requires supervision that captures valid tool interactions, but query-first synthesis can produce plausible requests that do not correspond to valid tool sequences, compatible parameters, or available data. We propose SyntheticAgentTraceQA, an execution-first framework that constructs workflow templates, assigns tools under data-flow constraints, executes and validates traces, and only then synthesizes user tasks, teacher reasoning, and reference answers. Across the evaluated tool ecosystems, fine-tuning Qwen3.5 variants on the resulting data improves several measures of tool-use behavior, reference-trace agreement, and answer generation. Masked supervision, which excludes reasoning tokens from the loss, yields higher answer-production rates than full supervision and does not consistently improve trace agreement, especially at the 9B scale.
Article Search
Article: asews26agenticdevmain-p185-p (type: Extended Abstract (2 pages)) doi:10.1145/3843282.3844446
AgenticDev '26: "Bug Composition: Triggering ..."
Bug Composition: Triggering Multiple Bugs with Agentic Driver Generation
Vishva Arasan, Mahima Chaudhary, Wendy Contreras Martinez, Md Shafiuzzaman, Achintya Desai, and Tevfik Bultan
(University of California at Santa Barbara, USA)
Security vulnerabilities caused by bugs in software do not exist in isolation. Attackers can chain vulnerabilities caused by multiple bugs together to achieve their goals. However, determining whether multiple vulnerabilities can be triggered during a single execution is a difficult problem. In this paper, we present an agentic pipeline for automatically discovering whether a single shared input can trigger multiple bugs. In particular, we introduce an agentic approach that automatically generates predicates and constraint-based harnesses from buggy code, leveraging symbolic execution and SMT solving. We validate our approach using real-world vulnerabilities across three widely used, open source libraries: tcpdump, libxml2, and nDPI. Our pipeline successfully generates concrete shared inputs to trigger multiple bug pairs, including heap buffer overflows in tcpdump, stack and heap memory corruptions in libxml2 and out-of-bounds reads across independent protocol dissectors in nDPI. Furthermore, our methodology correctly identifies and proves the absence of shared inputs when bugs require mutually exclusive constraints. Our work introduces and demonstrates the feasibility of automated bug composition analysis, results of which can be valuable in identifying potential exploit chains and prioritizing critical vulnerabilities for patching.
Article Search
Article: asews26agenticdevmain-p97-p (type: Full Paper) doi:10.1145/3843282.3844434
AgenticDev '26: "Defining and Deploying Agentic ..."
Defining and Deploying Agentic Swarms with the BESSER Low-Code Platform
Adem Ait, Javier Luis Cánovas Izquierdo, and Jordi Cabot
(University of Luxembourg, Luxembourg; Open University of Catalonia, Spain; Luxembourg Institute of Science and Technology, Luxembourg)
Complex software engineering tasks increasingly rely on agentic swarms, i.e., multi-agent systems in which specialized agents reason, plan, and act collaboratively to decompose a problem, propose solutions, review each other’s work, and converge on a decision. Today, such swarms are typically built directly in framework code, or with no-code tools that capture only the workflow, leaving per-agent capabilities and confidence, decision policies at merge points, and deployment without explicit, first-class abstractions. We argue that engineering agentic swarms calls for model-driven abstractions that span four complementary levels of granularity: the whole swarm, the process within a swarm, the merge point within a process, and the individual agent. In this demo we present an end-to-end approach on top of the open-source BESSER low-code platform that realizes these levels through four cross-referenced models, i.e., an agentic BPMN process, a state-machine-based Agent diagram, and UML Component and Deployment diagrams, authored in a web-based modeling editor. From the Deployment diagram, the fully specified swarm is generated and deployed to a Docker engine. We demonstrate the approach with a bug-fixing swarm composed of an Analyzer, two Solvers, and a Reviewer.
Article Search
Article: asews26agenticdevmain-p102-p (type: Short Paper) doi:10.1145/3843282.3844437
AgenticDev '26: "BDD-Bench: A Behavior-Driven ..."
BDD-Bench: A Behavior-Driven Benchmark for Coding Agents across the Application Lifecycle
Michał Jureczka, Anna Paleczek, Aiden Campbell, and Karol Przystalski
(Jagiellonian University, Poland; Exadel, Poland; AGH University of Krakow, Poland)
The main contribution of this paper is BDD-Bench, a repository-level benchmark in which coding agents implement executable Gherkin specifications across an application's development lifecycle. Its 111 chronologically ordered instances come from five Python repositories. In lifecycle mode, the agent solves consecutive instances on an evolving codebase, carrying each solution forward. Original tests are augmented with visible and hidden model-generated scenarios to strengthen behavioral coverage and assess generalization. Across thirteen models evaluated under a single agent harness with three runs each, the best-performing model, Claude Opus 4.8, resolves 45.0% of instances, covering 29.5% of the production code in the benchmark's reference patches when weighted by size. We further introduce a fine-grained failure taxonomy that pinpoints why lifecycle chains stall, and a matched-rerun design that isolates how agent-authored versus reference code histories shape an agent's downstream progress. These results show that even the strongest configurations often pass visible tests but fail withheld variations of the same behavioral specification.
Article Search
Article: asews26agenticdevmain-p25-p (type: Full Paper) doi:10.1145/3843282.3844426
AgenticDev '26: "Defining and Deploying Agentic ..."
Defining and Deploying Agentic Swarms with the BESSER Low-Code Platform
Adem Ait, Javier Luis Cánovas Izquierdo, and Jordi Cabot
(University of Luxembourg, Luxembourg; Open University of Catalonia, Spain; Luxembourg Institute of Science and Technology, Luxembourg)
Complex software engineering tasks increasingly rely on agentic swarms, i.e., multi-agent systems in which specialized agents reason, plan, and act collaboratively to decompose a problem, propose solutions, review each other’s work, and converge on a decision. Today, such swarms are typically built directly in framework code, or with no-code tools that capture only the workflow, leaving per-agent capabilities and confidence, decision policies at merge points, and deployment without explicit, first-class abstractions. We argue that engineering agentic swarms calls for model-driven abstractions that span four complementary levels of granularity: the whole swarm, the process within a swarm, the merge point within a process, and the individual agent. In this demo we present an end-to-end approach on top of the open-source BESSER low-code platform that realizes these levels through four cross-referenced models, i.e., an agentic BPMN process, a state-machine-based Agent diagram, and UML Component and Deployment diagrams, authored in a web-based modeling editor. From the Deployment diagram, the fully specified swarm is generated and deployed to a Docker engine. We demonstrate the approach with a bug-fixing swarm composed of an Analyzer, two Solvers, and a Reviewer.
Article Search
Article: asews26agenticdevmain-p102-p (type: Short Paper) doi:10.1145/3843282.3844437
AgenticDev '26: "Bug Composition: Triggering ..."
Bug Composition: Triggering Multiple Bugs with Agentic Driver Generation
Vishva Arasan, Mahima Chaudhary, Wendy Contreras Martinez, Md Shafiuzzaman, Achintya Desai, and Tevfik Bultan
(University of California at Santa Barbara, USA)
Security vulnerabilities caused by bugs in software do not exist in isolation. Attackers can chain vulnerabilities caused by multiple bugs together to achieve their goals. However, determining whether multiple vulnerabilities can be triggered during a single execution is a difficult problem. In this paper, we present an agentic pipeline for automatically discovering whether a single shared input can trigger multiple bugs. In particular, we introduce an agentic approach that automatically generates predicates and constraint-based harnesses from buggy code, leveraging symbolic execution and SMT solving. We validate our approach using real-world vulnerabilities across three widely used, open source libraries: tcpdump, libxml2, and nDPI. Our pipeline successfully generates concrete shared inputs to trigger multiple bug pairs, including heap buffer overflows in tcpdump, stack and heap memory corruptions in libxml2 and out-of-bounds reads across independent protocol dissectors in nDPI. Furthermore, our methodology correctly identifies and proves the absence of shared inputs when bugs require mutually exclusive constraints. Our work introduces and demonstrates the feasibility of automated bug composition analysis, results of which can be valuable in identifying potential exploit chains and prioritizing critical vulnerabilities for patching.
Article Search
Article: asews26agenticdevmain-p97-p (type: Full Paper) doi:10.1145/3843282.3844434
AgenticDev '26: "Extender: A Multi-agent System ..."
Extender: A Multi-agent System for Code Extensibility Review
Jiahe Chen and Jiacheng Shen
(Duke Kunshan University, China)
Software systems must evolve as requirements and environments change, which makes extensibility a central quality goal. The SOLID principles, five object-oriented design principles, are widely endorsed in software engineering practice to evaluate software quality. However, automatically detecting violations of SOLID principles remains difficult since many violations are latent design flaws that only surface when the code must be extended. In this paper, we propose to check the violations of SOLID principles by explicitly extending and checking the extensibility of existing code. Based on this idea, we design , a multi-agent system that detects SOLID violations by generating principle-specific extension requirements, asking AI agents to implement them, and checking whether the resulting diffs expose design defects. Experimental results show that achieves 87.9% detection accuracy with a Qwen3.5 9B model, which is 32.1 points higher than the state-of-the-art LLM-based approaches.
Article Search
Article: asews26agenticdevmain-p21-p (type: Short Paper) doi:10.1145/3843282.3844424
AgenticDev '26: "Execution-First Synthetic ..."
Execution-First Synthetic Tool-Use Trace Generation for LLM Agents
Hafsa Ouajdi, Francesco Giannuzzo, Alaa Boukhary, Paolo Papotti, Gerard Conangla, and Adam Elwood
(EURECOM, France; Aily Labs, Munich, Germany)
Agentic systems increasingly operate through executable workflows: they search artifacts, invoke tools, inspect structured observations, and query databases. Training such agents requires supervision that captures valid tool interactions, but query-first synthesis can produce plausible requests that do not correspond to valid tool sequences, compatible parameters, or available data. We propose SyntheticAgentTraceQA, an execution-first framework that constructs workflow templates, assigns tools under data-flow constraints, executes and validates traces, and only then synthesizes user tasks, teacher reasoning, and reference answers. Across the evaluated tool ecosystems, fine-tuning Qwen3.5 variants on the resulting data improves several measures of tool-use behavior, reference-trace agreement, and answer generation. Masked supervision, which excludes reasoning tokens from the loss, yields higher answer-production rates than full supervision and does not consistently improve trace agreement, especially at the 9B scale.
Article Search
Article: asews26agenticdevmain-p185-p (type: Extended Abstract (2 pages)) doi:10.1145/3843282.3844446
AgenticDev '26: "Bug Composition: Triggering ..."
Bug Composition: Triggering Multiple Bugs with Agentic Driver Generation
Vishva Arasan, Mahima Chaudhary, Wendy Contreras Martinez, Md Shafiuzzaman, Achintya Desai, and Tevfik Bultan
(University of California at Santa Barbara, USA)
Security vulnerabilities caused by bugs in software do not exist in isolation. Attackers can chain vulnerabilities caused by multiple bugs together to achieve their goals. However, determining whether multiple vulnerabilities can be triggered during a single execution is a difficult problem. In this paper, we present an agentic pipeline for automatically discovering whether a single shared input can trigger multiple bugs. In particular, we introduce an agentic approach that automatically generates predicates and constraint-based harnesses from buggy code, leveraging symbolic execution and SMT solving. We validate our approach using real-world vulnerabilities across three widely used, open source libraries: tcpdump, libxml2, and nDPI. Our pipeline successfully generates concrete shared inputs to trigger multiple bug pairs, including heap buffer overflows in tcpdump, stack and heap memory corruptions in libxml2 and out-of-bounds reads across independent protocol dissectors in nDPI. Furthermore, our methodology correctly identifies and proves the absence of shared inputs when bugs require mutually exclusive constraints. Our work introduces and demonstrates the feasibility of automated bug composition analysis, results of which can be valuable in identifying potential exploit chains and prioritizing critical vulnerabilities for patching.
Article Search
Article: asews26agenticdevmain-p97-p (type: Full Paper) doi:10.1145/3843282.3844434
AgenticDev '26: "Who Should Own the Loop? Harness ..."
Who Should Own the Loop? Harness Decomposition for Small-Model Repository Repair
Francesco Dente, Dario Satriani, Donatello Santoro, Enzo Veltri, and Paolo Papotti
(EURECOM, France; University of Basilicata, Italy)
Repository-level software repair is commonly approached by placing a language model in a free-form loop: the model explores the codebase, runs commands, edits files, interprets failures, and decides when the issue is resolved. This design works well for highly capable models, but it places many responsibilities on the model at once. When a smaller model fails, it is therefore difficult to tell whether it cannot produce the required patch, or whether it becomes lost while organizing the steps that lead to it. This paper studies an alternative design in which the repair process is decomposed into explicit stages. In particular, we separate finding the relevant code, constructing and using an executable signal of the bug, and generating the final edit. This perspective treats the harness not as a passive wrapper, but as part of the system being designed. We evaluate eight open-weight models from two families, ranging from 0.8B to 31B parameters, on a subset of SWE-bench Verified. Our results show that harness-managed control flow can substantially improve the effectiveness of the smallest models. For example, Gemma~4 E2B improves from 5.0% in the free-form loop to 27.4% in the decomposed repair loop and Gemma~4 E4B from 14.6% to 45.7%. Because the stages interact only through the context they build, they need not all be performed by the same model: handing localization and reproduction to stronger open-weight models, while the small model remains responsible only for the edit, raises the same two models further, to 39.8% and 53.7%. These findings suggest that small models can be useful for repository repair when responsibilities are divided across explicit stages that can be independently assigned to the component best suited to each.
Article Search
Article: asews26agenticdevmain-p131-p (type: Short Paper) doi:10.1145/3843282.3844442
AgenticDev '26: "Bug Composition: Triggering ..."
Bug Composition: Triggering Multiple Bugs with Agentic Driver Generation
Vishva Arasan, Mahima Chaudhary, Wendy Contreras Martinez, Md Shafiuzzaman, Achintya Desai, and Tevfik Bultan
(University of California at Santa Barbara, USA)
Security vulnerabilities caused by bugs in software do not exist in isolation. Attackers can chain vulnerabilities caused by multiple bugs together to achieve their goals. However, determining whether multiple vulnerabilities can be triggered during a single execution is a difficult problem. In this paper, we present an agentic pipeline for automatically discovering whether a single shared input can trigger multiple bugs. In particular, we introduce an agentic approach that automatically generates predicates and constraint-based harnesses from buggy code, leveraging symbolic execution and SMT solving. We validate our approach using real-world vulnerabilities across three widely used, open source libraries: tcpdump, libxml2, and nDPI. Our pipeline successfully generates concrete shared inputs to trigger multiple bug pairs, including heap buffer overflows in tcpdump, stack and heap memory corruptions in libxml2 and out-of-bounds reads across independent protocol dissectors in nDPI. Furthermore, our methodology correctly identifies and proves the absence of shared inputs when bugs require mutually exclusive constraints. Our work introduces and demonstrates the feasibility of automated bug composition analysis, results of which can be valuable in identifying potential exploit chains and prioritizing critical vulnerabilities for patching.
Article Search
Article: asews26agenticdevmain-p97-p (type: Full Paper) doi:10.1145/3843282.3844434
AgenticDev '26: "Value-Preserving Architectures ..."
Value-Preserving Architectures for Agentic AI Systems
Alessandro Pesare, Tommaso Dolci, Katja Hose, and Emanuel Sallinger
(TU Wien, Austria)
The emergence of agentic AI and LLM-based multi-agent systems (MAS) presents unprecedented opportunities for automating complex tasks, while simultaneously raising critical concerns about the preservation of fundamental human-centered values, such as privacy, fairness, and safety. Although software engineering has traditionally focused on functional correctness, the adoption of LLMs and AI agents into complex socio-technical systems has intensified the need for responsible software engineering and robust value alignment. In MAS, architectural design decisions, such as coordination mechanisms, communication protocols, and system topologies, play a central role in shaping system behavior and the outcomes they produce. This paper argues that architectural choices influence not only the functionality and performance of MAS but can also promote value-oriented system behavior. Therefore, we investigate how different architectural designs support different human-centered values, discussing the following value-preserving architectural patterns: (i) a privacy-aware architecture with a federated topology, (ii) a distributed architecture to promote pluralism and diversity, and (iii) a guard-agent architecture to detect and mitigate unfairness. Finally, we introduce representative use cases to illustrate the proposed architectures in real-world scenarios. By linking architectural design with human-centered values, this work lays the foundation for a unified set of architectural patterns and guidelines towards the design of trustworthy MAS.
Article Search
Article: asews26agenticdevmain-p100-p (type: Short Paper) doi:10.1145/3843282.3844436
AgenticDev '26: "Execution-First Synthetic ..."
Execution-First Synthetic Tool-Use Trace Generation for LLM Agents
Hafsa Ouajdi, Francesco Giannuzzo, Alaa Boukhary, Paolo Papotti, Gerard Conangla, and Adam Elwood
(EURECOM, France; Aily Labs, Munich, Germany)
Agentic systems increasingly operate through executable workflows: they search artifacts, invoke tools, inspect structured observations, and query databases. Training such agents requires supervision that captures valid tool interactions, but query-first synthesis can produce plausible requests that do not correspond to valid tool sequences, compatible parameters, or available data. We propose SyntheticAgentTraceQA, an execution-first framework that constructs workflow templates, assigns tools under data-flow constraints, executes and validates traces, and only then synthesizes user tasks, teacher reasoning, and reference answers. Across the evaluated tool ecosystems, fine-tuning Qwen3.5 variants on the resulting data improves several measures of tool-use behavior, reference-trace agreement, and answer generation. Masked supervision, which excludes reasoning tokens from the loss, yields higher answer-production rates than full supervision and does not consistently improve trace agreement, especially at the 9B scale.
Article Search
Article: asews26agenticdevmain-p185-p (type: Extended Abstract (2 pages)) doi:10.1145/3843282.3844446
AgenticDev '26: "An Evidence-Preserving Architecture ..."
An Evidence-Preserving Architecture for LLM-Assisted Vulnerability Review of Robotics Software
Anhtuan Le, Gregory Epiphaniou, and Carsten Maple
(University of Warwick, UK)
In industrial robotics, proprietary source often cannot leave the customer’s premises. Suppliers may therefore pair a static anal-yser with a large language model (LLM) running locally and send both outputs to one review queue. Combining the outputs can obscure two facts: which component supplied the evidence for a finding and which component may close it. We present an evidencepreserving architecture that keeps both facts visible. Each detector’s output remains a separate record, duplicates are linked rather than deleted, and every queue names the policy that derived it. A model’s negative verdict cannot deactivate another component’s finding. We instantiate the architecture for ROS 2 and C++ and evaluate it on 102 source windows. These include 40 vulnerable cases, each paired with a repaired twin. Seven local LLMs were run three times per case, and alternative merge, authority and invocation policies were replayed over the stored evidence. The results show that (1) conventional merging preserved coverage but erased most model attribution; (2) allowing a model verdict to close a case removed up to 15 confirmed issues; (3) the models rarely distinguished a defect from its repair; (4) retrieval added context but not accuracy because it could not reach a weakness class that the model had not proposed; and (5) 34% of model–issue detections were inconsistent across the three runs. Compact local LLMs can prioritise review and supply traceable evidence, but should not have authority to dismiss a security finding on their own.
Article Search
Article: asews26agenticdevmain-p99-p (type: Full Paper) doi:10.1145/3843282.3844435
AgenticDev '26: "Teach-to-Crash: A Closed-Loop ..."
Teach-to-Crash: A Closed-Loop Student–Teacher LLM Framework for Collision-Inducing Test Scenario Generation
Zaid Ghazal, Khouloud Gaaloul, and Bruce Maxim
(University of Michigan at Dearborn, USA)
Validating Autonomous Driving Systems (ADS) in simulation requires testing architectures that can discover rare, safety-critical failures while generating scenarios that are executable, diverse, and useful for downstream failure analysis. We introduce Teach-to-Crash, a closed-loop testing framework that combines a constrained ego-centric scenario representation, stagnation-aware search control, and a dual-LLM architecture for adaptive failure discovery. A high-reasoning Teacher LLM acts as an adaptive search controller, while a low-reasoning Student LLM emits simulator-executable scenarios in a strict JSON schema. The Teacher intervenes only when rolling collision rate and time-to-collision metrics stagnate, providing strategic guidance to redirect the search. In a CARLA case study with two experimental setups that vary the ego vehicle’s speed policy, Teach-to-Crash achieves the highest Collision Hit Rate (90.79%), the shortest mean Time-to-Collision (18.31s), and a competitive Collision Discovery Rate (136.21). PAFOT attains a higher mean CDR (179.44), but with substantially larger variance. Teach-to-Crash also yields the highest diversity (0.547) and, averaged across both setups on the CARLA Traffic Manager controller, the highest avoidability-based usefulness proxy (60.04%) among the compared methods. These results, within the evaluated CARLA scope, provide evidence that closed-loop dual-LLM reasoning can steer adversarial simulation-based testing over a constrained executable program space, generating failures that are frequent, structurally diverse, and assessed as more frequently avoidable.
Article Search
Artifacts Available
Article: asews26agenticdevmain-p126-p (type: Full Paper) doi:10.1145/3843282.3844441
AgenticDev '26: "Looping Is Not Reliability: ..."
Looping Is Not Reliability: State-Bound Evidence and Typed Revision Contracts for Agentic Code Repair
Xueping Gao, Jianwei Yang, and Qiang Yang
(Alibaba Cloud Computing, China)
Generate–test–revise loops are common in coding agents, but repetition alone provides no reliability guarantee. We study the gap between finding a correct patch and retaining, verifying, and submitting it. A sealed five-seed study over 30 HumanEval repairs produces 900 three-revision trajectories. Under forced revision, current correctness with current traces falls from 0.820 after one revision to 0.673 after two, although ever-correct rises to 0.847. Two common-state studies use 2,430 branches from identical frozen programs to remove post-treatment risk-set bias. In a prespecified 14B replication, stale traces harm 34/135 correct starts versus 4/135 with current traces, a 22.2-point increase (task-cluster 95% CI [8.9,37.0], exact Holm p=0.0337). A prospective 540-rollout policy eliminates observed correct-start harm but reduces wrong-start repair and fails its joint criterion. Repository experiments over 24 bugs and four coder stacks expose floor effects and component heterogeneity without Holm-significant effects. We therefore separate admission, preservation, grounded certification, competence, and liveness. We derive an evidence-bound typed loop contract and instantiate its mechanically enforceable subset in a reference implementation that binds verifier evidence to exact code states, preserves verified checkpoints, and emits auditable admission receipts. The implementation is an executable specification and conformance artifact, not evidence of improved repair competence or calibrated verifier dependence.
Article Search
Article: asews26agenticdevmain-p114-p (type: Full Paper) doi:10.1145/3843282.3844438
AgenticDev '26: "MOSAICO: A Multi-agent Infrastructure ..."
MOSAICO: A Multi-agent Infrastructure for Collaborative LLM-Powered Software Engineering
Tuong Manh Vu, Ionut Predoaia, Konstantinos Barmpis, Dimitris Kolovos, and Antonio García-Domínguez
(University of York, UK)
Modern software engineering tasks increasingly call for Artificial Intelligence (AI) assistance that goes beyond a single large language model (LLM) acting alone. We present MOSAICO, an infrastructure for building and orchestrating communities of LLM-powered agents that collaborate to solve software engineering tasks. MOSAICO provides a reference client, a reference agent, and a set of task-specialised agent roles (solution, supervision, and consensus agents) that a collaboration agent invokes according to a workflow to complete a task. We showcase MOSAICO through three concrete demonstrations: a metamodelling use case, where a team of four agents jointly construct and validate a domain metamodel; a UML generation use case, where an agent converts a diagram image into textual notation; and a code documentation use case, where an agent uses client-side tools to generate documentation for existing code. Together, these demonstrations illustrate the generality of MOSAICO: a single, role-based agent infrastructure can support structurally different software engineering tasks with minimal reconfiguration.
Article Search
Article: asews26agenticdevmain-p159-p (type: Short Paper) doi:10.1145/3843282.3844444
AgenticDev '26: "Towards Model-Driven Generation ..."
Towards Model-Driven Generation of Domain-Specific Modelling Agents for Local Language Models
Antonio Garcia-Dominguez, Ionut Predoaia, Tuong Manh Vu, Konstantinos Barmpis, and Dimitris Kolovos
(University of York, UK)
Current frontier LLMs leverage vast training datasets with knowledge about a broad range of programming languages and frameworks, and can often solve tasks with minimal prompting. However, there are scenarios where remote LLMs are unsuitable, due to cost, privacy, security, or intellectual property concerns, and local language models need to be used. These smaller local language models typically require more careful guidance in order to achieve acceptable performance. In this paper, we evaluate the impact of using local models (in the 14-31B range) for object-oriented modelling, and we propose an approach for partially automating the creation of agents that "think in code" to solve such problems, instead of directly producing an output. We compare the direct invocation of Claude Sonnet 4.6 with that of several open-weight models, and with the use of those open-weight models while "thinking in code" as proposed. We observe that the best open-weight models, when "thinking in code", can produce models with similar Jaccard similarity scores to the model solutions in the Golden UML ModelSet dataset as Sonnet, although at the cost of increased token consumption. Based on the observed potential for generating domain-specific agents that reduce the demands on the LLM and therefore on the hardware requirements, we set out several lines of further work.
Article Search
Artifacts Available
Article: asews26agenticdevmain-p58-p (type: Full Paper) doi:10.1145/3843282.3844429
AgenticDev '26: "Teach-to-Crash: A Closed-Loop ..."
Teach-to-Crash: A Closed-Loop Student–Teacher LLM Framework for Collision-Inducing Test Scenario Generation
Zaid Ghazal, Khouloud Gaaloul, and Bruce Maxim
(University of Michigan at Dearborn, USA)
Validating Autonomous Driving Systems (ADS) in simulation requires testing architectures that can discover rare, safety-critical failures while generating scenarios that are executable, diverse, and useful for downstream failure analysis. We introduce Teach-to-Crash, a closed-loop testing framework that combines a constrained ego-centric scenario representation, stagnation-aware search control, and a dual-LLM architecture for adaptive failure discovery. A high-reasoning Teacher LLM acts as an adaptive search controller, while a low-reasoning Student LLM emits simulator-executable scenarios in a strict JSON schema. The Teacher intervenes only when rolling collision rate and time-to-collision metrics stagnate, providing strategic guidance to redirect the search. In a CARLA case study with two experimental setups that vary the ego vehicle’s speed policy, Teach-to-Crash achieves the highest Collision Hit Rate (90.79%), the shortest mean Time-to-Collision (18.31s), and a competitive Collision Discovery Rate (136.21). PAFOT attains a higher mean CDR (179.44), but with substantially larger variance. Teach-to-Crash also yields the highest diversity (0.547) and, averaged across both setups on the CARLA Traffic Manager controller, the highest avoidability-based usefulness proxy (60.04%) among the compared methods. These results, within the evaluated CARLA scope, provide evidence that closed-loop dual-LLM reasoning can steer adversarial simulation-based testing over a constrained executable program space, generating failures that are frequent, structurally diverse, and assessed as more frequently avoidable.
Article Search
Artifacts Available
Article: asews26agenticdevmain-p126-p (type: Full Paper) doi:10.1145/3843282.3844441
AgenticDev '26: "Execution-First Synthetic ..."
Execution-First Synthetic Tool-Use Trace Generation for LLM Agents
Hafsa Ouajdi, Francesco Giannuzzo, Alaa Boukhary, Paolo Papotti, Gerard Conangla, and Adam Elwood
(EURECOM, France; Aily Labs, Munich, Germany)
Agentic systems increasingly operate through executable workflows: they search artifacts, invoke tools, inspect structured observations, and query databases. Training such agents requires supervision that captures valid tool interactions, but query-first synthesis can produce plausible requests that do not correspond to valid tool sequences, compatible parameters, or available data. We propose SyntheticAgentTraceQA, an execution-first framework that constructs workflow templates, assigns tools under data-flow constraints, executes and validates traces, and only then synthesizes user tasks, teacher reasoning, and reference answers. Across the evaluated tool ecosystems, fine-tuning Qwen3.5 variants on the resulting data improves several measures of tool-use behavior, reference-trace agreement, and answer generation. Masked supervision, which excludes reasoning tokens from the loss, yields higher answer-production rates than full supervision and does not consistently improve trace agreement, especially at the 9B scale.
Article Search
Article: asews26agenticdevmain-p185-p (type: Extended Abstract (2 pages)) doi:10.1145/3843282.3844446
AgenticDev '26: "Spec-Graph Meta-framework ..."
Spec-Graph Meta-framework for AI-Assisted Software Development: An Industry Case
Daniel Soto, Daniel González-Rueda, Mario Linares-Vásquez, and Andrés Hurtado
(Tres Astronautas, Colombia; Universidad de los Andes, Colombia; Blend 360, USA)
AI-assisted software development increasingly relies on large lan- guage models to transform natural-language specifications into code. However, in most current workflows, specifications are treated as inline prompt content, limiting their structure, reuse, and trace- ability across development tasks. This paper introduces a meta- framework for spec-driven AI-assisted development in which spec- ifications are represented as a typed, queryable graph consumed at runtime by software agents. The framework is organized around three principles: legibility, achieved through a typed hierarchy that distinguishes buildable artifacts from contextual constraints; addressability, enabled by a minimal set of edge types, substrate- agnostic graph materialization, and a cost-function-driven sequenc- ing mechanism; and longevity, maintained through continuous cu- ration of the specification substrate. We instantiate the framework in an industrial setting involving a multi-project software develop- ment organization with over fifty employees, using thirteen speci- fication types, a multi-substrate architecture, and four cooperating AI agents. To explore the proposed framework’s behavior in prac- tice we surveyed 19 practitioners in the organization after using the framework from one to five months.
Article Search
Article: asews26agenticdevmain-p86-p (type: Full Paper) doi:10.1145/3843282.3844432
AgenticDev '26: "Value-Preserving Architectures ..."
Value-Preserving Architectures for Agentic AI Systems
Alessandro Pesare, Tommaso Dolci, Katja Hose, and Emanuel Sallinger
(TU Wien, Austria)
The emergence of agentic AI and LLM-based multi-agent systems (MAS) presents unprecedented opportunities for automating complex tasks, while simultaneously raising critical concerns about the preservation of fundamental human-centered values, such as privacy, fairness, and safety. Although software engineering has traditionally focused on functional correctness, the adoption of LLMs and AI agents into complex socio-technical systems has intensified the need for responsible software engineering and robust value alignment. In MAS, architectural design decisions, such as coordination mechanisms, communication protocols, and system topologies, play a central role in shaping system behavior and the outcomes they produce. This paper argues that architectural choices influence not only the functionality and performance of MAS but can also promote value-oriented system behavior. Therefore, we investigate how different architectural designs support different human-centered values, discussing the following value-preserving architectural patterns: (i) a privacy-aware architecture with a federated topology, (ii) a distributed architecture to promote pluralism and diversity, and (iii) a guard-agent architecture to detect and mitigate unfairness. Finally, we introduce representative use cases to illustrate the proposed architectures in real-world scenarios. By linking architectural design with human-centered values, this work lays the foundation for a unified set of architectural patterns and guidelines towards the design of trustworthy MAS.
Article Search
Article: asews26agenticdevmain-p100-p (type: Short Paper) doi:10.1145/3843282.3844436
AgenticDev '26: "Spec-Graph Meta-framework ..."
Spec-Graph Meta-framework for AI-Assisted Software Development: An Industry Case
Daniel Soto, Daniel González-Rueda, Mario Linares-Vásquez, and Andrés Hurtado
(Tres Astronautas, Colombia; Universidad de los Andes, Colombia; Blend 360, USA)
AI-assisted software development increasingly relies on large lan- guage models to transform natural-language specifications into code. However, in most current workflows, specifications are treated as inline prompt content, limiting their structure, reuse, and trace- ability across development tasks. This paper introduces a meta- framework for spec-driven AI-assisted development in which spec- ifications are represented as a typed, queryable graph consumed at runtime by software agents. The framework is organized around three principles: legibility, achieved through a typed hierarchy that distinguishes buildable artifacts from contextual constraints; addressability, enabled by a minimal set of edge types, substrate- agnostic graph materialization, and a cost-function-driven sequenc- ing mechanism; and longevity, maintained through continuous cu- ration of the specification substrate. We instantiate the framework in an industrial setting involving a multi-project software develop- ment organization with over fifty employees, using thirteen speci- fication types, a multi-substrate architecture, and four cooperating AI agents. To explore the proposed framework’s behavior in prac- tice we surveyed 19 practitioners in the organization after using the framework from one to five months.
Article Search
Article: asews26agenticdevmain-p86-p (type: Full Paper) doi:10.1145/3843282.3844432
AgenticDev '26: "Cost-Effective Repository ..."
Cost-Effective Repository Exploration for Agentic Issue Localization
Mohammad Nour Al Awad and Sergey Ivanov
(ITMO University, Russian Federation)
Repository exploration is a distinct and costly stage of coding-agent pipelines: before generating a patch, an agent must identify which repository files are likely to matter. We study whether this stage can be delegated to lower-cost models while retaining useful localization quality. Using our IssueLoc-Bench, we evaluate five explorer models under the same read-only interactive interface on 499 SWE-bench Verified-derived tasks and 500 tasks from 153 additional repositories. We measure early candidate discovery, top-three gold-file coverage, strict file-set recovery, agent time, and token usage, with paired instance-level uncertainty and repository-clustered sensitivity analysis. The highest-quality explorer leads across localization metrics, but substantially cheaper operating points emerge: depending on the model and evaluation arm, lower-cost explorers retain approximately 78–94% of the reference Hit@3 and 73–92% of its F1 while reducing mean agent time by 41–88% and token usage by 84–95%. The preferred operating point depends on how localization is consumed downstream: ranking and coverage metrics characterize recoverable candidate handoffs, whereas F1 and exact match characterize restrictive file gates. These results support treating repository exploration as an independently measurable and budgetable stage of modular coding agents, with explorer selection guided by the downstream handoff contract.
Article Search
Article: asews26agenticdevmain-p18-p (type: Full Paper) doi:10.1145/3843282.3844422
AgenticDev '26: "BDD-Bench: A Behavior-Driven ..."
BDD-Bench: A Behavior-Driven Benchmark for Coding Agents across the Application Lifecycle
Michał Jureczka, Anna Paleczek, Aiden Campbell, and Karol Przystalski
(Jagiellonian University, Poland; Exadel, Poland; AGH University of Krakow, Poland)
The main contribution of this paper is BDD-Bench, a repository-level benchmark in which coding agents implement executable Gherkin specifications across an application's development lifecycle. Its 111 chronologically ordered instances come from five Python repositories. In lifecycle mode, the agent solves consecutive instances on an evolving codebase, carrying each solution forward. Original tests are augmented with visible and hidden model-generated scenarios to strengthen behavioral coverage and assess generalization. Across thirteen models evaluated under a single agent harness with three runs each, the best-performing model, Claude Opus 4.8, resolves 45.0% of instances, covering 29.5% of the production code in the benchmark's reference patches when weighted by size. We further introduce a fine-grained failure taxonomy that pinpoints why lifecycle chains stall, and a matched-rerun design that isolates how agent-authored versus reference code histories shape an agent's downstream progress. These results show that even the strongest configurations often pass visible tests but fail withheld variations of the same behavioral specification.
Article Search
Article: asews26agenticdevmain-p25-p (type: Full Paper) doi:10.1145/3843282.3844426
AgenticDev '26: "Agents as Architecture Evaluators: ..."
Agents as Architecture Evaluators: Automating ATAM and SAAM for Continuous Assessment
Adyansh Kakran, Sathvika Miryala, and Karthik Vaidhyanathan
(IIIT Hyderabad, India)
The long-term success of a software system relies heavily on its un-derlying architecture, which dictates its performance with respect to quality attributes. Software Architecture Evaluation methods, such as the Architecture Tradeoff Analysis Method (ATAM) and the Software Architecture Analysis Method (SAAM), are widely recognized for their ability to mitigate risks and ensure reliable quality attributes. Their adoption remains limited due to cost, time, and expertise. We investigate agentic AI to lower these barriers. From human-conducted evaluations in the literature, we built a dataset of 39 cases and an agentic workflow that synthesizes evaluation reports from high-level architectural descriptions. Automated metrics and human evaluation yield relevance and completeness of 4.5/5 and 3.9/5. Agents automated scenario generation and risk identification, under 5 minutes at less than $0.50 per run.
Article Search
Article: asews26agenticdevmain-p78-p (type: Extended Abstract (2 pages)) doi:10.1145/3843282.3844431
AgenticDev '26: "Context Inheritance: A Git-Native ..."
Context Inheritance: A Git-Native Architecture and Pre-registered Study of Repository Context for AI Coding Agents
Pavel Kerbel, Milana Kerbel, Vitali Abramov, and Liat Abramov
(Metatron Research, Israel)
AI coding agents repeatedly rediscover project constraints, because repositories version code but not operating knowledge: the decisions taken, the alternatives rejected, the constraints found during execution. We introduce the Repository Context Layer (RCL) — the repository carries its own agent-facing context, versioned with the code, deterministically discoverable, consulted by contract, and writable by agents only under human review, in a minimal default format, context.md — and study, in a pre-registered experiment on SWE-bench Verified, when it helps. The organizing observation is a capability-dependent author–reader pattern we call context inheritance: repository context behaves like senior knowledge, and across the levels we observe its value flows toward weaker readers. A small local executor gains a large navigational benefit from context authored by a more capable model (+26 pp localization on topic-matched tasks, $p < 0.0001$) but nothing from context it wrote itself (+0.0 pp); with two author levels drawn from different model families, we report this as a pattern consistent with a capability gradient rather than an isolated causal variable. At the frontier the axes shift: an agent appears to gain from its own failure-derived lessons, but not from answer-derived ones (+14.6 pp resolve on constraint-sharing tasks; $p = 0.041$ uncorrected, $0.081$ after Holm correction within its registered family, so we report the direction rather than a confirmed effect; −32% tokens per resolved task); how context is delivered matters (exploratorily, a sharded store the agent reads selectively outperformed a monolith at a third of the tokens at the local tier); and on a broad random sample a frontier agent sits at a resolve ceiling that no context improves. The design consequence is concrete: version repository context as a first-class artifact, author it with the strongest writer available, and expect the benefit in cheaper agents that read it.
Article Search
Article: asews26agenticdevmain-p22-p (type: Full Paper) doi:10.1145/3843282.3844425
AgenticDev '26: "Context Inheritance: A Git-Native ..."
Context Inheritance: A Git-Native Architecture and Pre-registered Study of Repository Context for AI Coding Agents
Pavel Kerbel, Milana Kerbel, Vitali Abramov, and Liat Abramov
(Metatron Research, Israel)
AI coding agents repeatedly rediscover project constraints, because repositories version code but not operating knowledge: the decisions taken, the alternatives rejected, the constraints found during execution. We introduce the Repository Context Layer (RCL) — the repository carries its own agent-facing context, versioned with the code, deterministically discoverable, consulted by contract, and writable by agents only under human review, in a minimal default format, context.md — and study, in a pre-registered experiment on SWE-bench Verified, when it helps. The organizing observation is a capability-dependent author–reader pattern we call context inheritance: repository context behaves like senior knowledge, and across the levels we observe its value flows toward weaker readers. A small local executor gains a large navigational benefit from context authored by a more capable model (+26 pp localization on topic-matched tasks, $p < 0.0001$) but nothing from context it wrote itself (+0.0 pp); with two author levels drawn from different model families, we report this as a pattern consistent with a capability gradient rather than an isolated causal variable. At the frontier the axes shift: an agent appears to gain from its own failure-derived lessons, but not from answer-derived ones (+14.6 pp resolve on constraint-sharing tasks; $p = 0.041$ uncorrected, $0.081$ after Holm correction within its registered family, so we report the direction rather than a confirmed effect; −32% tokens per resolved task); how context is delivered matters (exploratorily, a sharded store the agent reads selectively outperformed a monolith at a third of the tokens at the local tier); and on a broad random sample a frontier agent sits at a resolve ceiling that no context improves. The design consequence is concrete: version repository context as a first-class artifact, author it with the strongest writer available, and expect the benefit in cheaper agents that read it.
Article Search
Article: asews26agenticdevmain-p22-p (type: Full Paper) doi:10.1145/3843282.3844425
AgenticDev '26: "They Do Not Fire Late, They ..."
They Do Not Fire Late, They Barely Fire: Four Surface-Behavior Stopping Baselines for Long-Horizon Coding Agents, and How to Report One
Simarjot Khanna
(Independent, Canada)
Long-horizon coding agents frequently waste budget by acting without progressing. A growing family of stop-signals proposes to abort such runs early, but each is evaluated on its own corpus against its own baseline, leaving their limits unknown. We operationalize four stop-signal families from published designs as inexpensive fixed-threshold detectors and evaluate them by replaying public agent trajectories; every claim is scoped to these author-built operationalizations, not to the published systems themselves. A retrospective Oracle Stop, the last step at which a run moved closer to the benchmark's reference patch, grounds the timing and waste analyses; the headline results need only run outcomes and detector fire steps. Held to a strict false-abort budget of 5%, these detectors fail not by firing late but by remaining silent on the large majority of doomed runs. The diagnosis replicated on four preregistered held-out configurations, and a threshold calibrated on one deployment either fell silent or violated the budget on others. The mechanism is that doomed and recoverable runs produce largely overlapping alarm-score distributions, and their separability stops improving by mid-run, so waiting does not sharpen the signal; in a difficulty-matched sample, task difficulty does not produce the separation. From this we derive a falsifiable separation target for future detectors, and a three-part reporting protocol: coverage at a stated false-abort budget, run-level separation, and whether the operating threshold survives a change of deployment.
Article Search
Article: asews26agenticdevmain-p195-p (type: Full Paper) doi:10.1145/3843282.3844447
AgenticDev '26: "MOSAICO: A Multi-agent Infrastructure ..."
MOSAICO: A Multi-agent Infrastructure for Collaborative LLM-Powered Software Engineering
Tuong Manh Vu, Ionut Predoaia, Konstantinos Barmpis, Dimitris Kolovos, and Antonio García-Domínguez
(University of York, UK)
Modern software engineering tasks increasingly call for Artificial Intelligence (AI) assistance that goes beyond a single large language model (LLM) acting alone. We present MOSAICO, an infrastructure for building and orchestrating communities of LLM-powered agents that collaborate to solve software engineering tasks. MOSAICO provides a reference client, a reference agent, and a set of task-specialised agent roles (solution, supervision, and consensus agents) that a collaboration agent invokes according to a workflow to complete a task. We showcase MOSAICO through three concrete demonstrations: a metamodelling use case, where a team of four agents jointly construct and validate a domain metamodel; a UML generation use case, where an agent converts a diagram image into textual notation; and a code documentation use case, where an agent uses client-side tools to generate documentation for existing code. Together, these demonstrations illustrate the generality of MOSAICO: a single, role-based agent infrastructure can support structurally different software engineering tasks with minimal reconfiguration.
Article Search
Article: asews26agenticdevmain-p159-p (type: Short Paper) doi:10.1145/3843282.3844444
AgenticDev '26: "Towards Model-Driven Generation ..."
Towards Model-Driven Generation of Domain-Specific Modelling Agents for Local Language Models
Antonio Garcia-Dominguez, Ionut Predoaia, Tuong Manh Vu, Konstantinos Barmpis, and Dimitris Kolovos
(University of York, UK)
Current frontier LLMs leverage vast training datasets with knowledge about a broad range of programming languages and frameworks, and can often solve tasks with minimal prompting. However, there are scenarios where remote LLMs are unsuitable, due to cost, privacy, security, or intellectual property concerns, and local language models need to be used. These smaller local language models typically require more careful guidance in order to achieve acceptable performance. In this paper, we evaluate the impact of using local models (in the 14-31B range) for object-oriented modelling, and we propose an approach for partially automating the creation of agents that "think in code" to solve such problems, instead of directly producing an output. We compare the direct invocation of Claude Sonnet 4.6 with that of several open-weight models, and with the use of those open-weight models while "thinking in code" as proposed. We observe that the best open-weight models, when "thinking in code", can produce models with similar Jaccard similarity scores to the model solutions in the Golden UML ModelSet dataset as Sonnet, although at the cost of increased token consumption. Based on the observed potential for generating domain-specific agents that reduce the demands on the LLM and therefore on the hardware requirements, we set out several lines of further work.
Article Search
Artifacts Available
Article: asews26agenticdevmain-p58-p (type: Full Paper) doi:10.1145/3843282.3844429
AgenticDev '26: "An Evidence-Preserving Architecture ..."
An Evidence-Preserving Architecture for LLM-Assisted Vulnerability Review of Robotics Software
Anhtuan Le, Gregory Epiphaniou, and Carsten Maple
(University of Warwick, UK)
In industrial robotics, proprietary source often cannot leave the customer’s premises. Suppliers may therefore pair a static anal-yser with a large language model (LLM) running locally and send both outputs to one review queue. Combining the outputs can obscure two facts: which component supplied the evidence for a finding and which component may close it. We present an evidencepreserving architecture that keeps both facts visible. Each detector’s output remains a separate record, duplicates are linked rather than deleted, and every queue names the policy that derived it. A model’s negative verdict cannot deactivate another component’s finding. We instantiate the architecture for ROS 2 and C++ and evaluate it on 102 source windows. These include 40 vulnerable cases, each paired with a repaired twin. Seven local LLMs were run three times per case, and alternative merge, authority and invocation policies were replayed over the stored evidence. The results show that (1) conventional merging preserved coverage but erased most model attribution; (2) allowing a model verdict to close a case removed up to 15 confirmed issues; (3) the models rarely distinguished a defect from its repair; (4) retrieval added context but not accuracy because it could not reach a weakness class that the model had not proposed; and (5) 34% of model–issue detections were inconsistent across the three runs. Compact local LLMs can prioritise review and supply traceable evidence, but should not have authority to dismiss a security finding on their own.
Article Search
Article: asews26agenticdevmain-p99-p (type: Full Paper) doi:10.1145/3843282.3844435
AgenticDev '26: "Spec-Graph Meta-framework ..."
Spec-Graph Meta-framework for AI-Assisted Software Development: An Industry Case
Daniel Soto, Daniel González-Rueda, Mario Linares-Vásquez, and Andrés Hurtado
(Tres Astronautas, Colombia; Universidad de los Andes, Colombia; Blend 360, USA)
AI-assisted software development increasingly relies on large lan- guage models to transform natural-language specifications into code. However, in most current workflows, specifications are treated as inline prompt content, limiting their structure, reuse, and trace- ability across development tasks. This paper introduces a meta- framework for spec-driven AI-assisted development in which spec- ifications are represented as a typed, queryable graph consumed at runtime by software agents. The framework is organized around three principles: legibility, achieved through a typed hierarchy that distinguishes buildable artifacts from contextual constraints; addressability, enabled by a minimal set of edge types, substrate- agnostic graph materialization, and a cost-function-driven sequenc- ing mechanism; and longevity, maintained through continuous cu- ration of the specification substrate. We instantiate the framework in an industrial setting involving a multi-project software develop- ment organization with over fifty employees, using thirteen speci- fication types, a multi-substrate architecture, and four cooperating AI agents. To explore the proposed framework’s behavior in prac- tice we surveyed 19 practitioners in the organization after using the framework from one to five months.
Article Search
Article: asews26agenticdevmain-p86-p (type: Full Paper) doi:10.1145/3843282.3844432
AgenticDev '26: "katIA: Hybrid Source Code ..."
katIA: Hybrid Source Code Change Impact Analysis
Katharina Stengg, Christian Macho, and Martin Pinzger
(University of Klagenfurt, Austria)
Maintaining software requires developers to identify the code entities affected by code changes. Existing approaches to change impact analysis (CIA) lack either high precision or high recall.Moreover, recent approaches are evaluated on benchmarks without labels for the relationships between changed code entities and therefore rely on simplified assumptions to determine these relationships. We propose katIA, a preliminary approach that combines static code analysis with LLM-based refinement for CIA. We implement two variants of katIA and evaluate them on a curated dataset. Our preliminary results suggest that both variants of katIA can improve CIA, with katIA-MA showing the strongest overall performance in our evaluation. These findings provide initial evidence that combining static code analysis and LLMs can benefit CIA.
Article Search
Article: asews26agenticdevmain-p2-p (type: Extended Abstract (2 pages)) doi:10.1145/3843282.3844420
AgenticDev '26: "Before the First Model Call: ..."
Before the First Model Call: Factorizing Token Alignment and Cue Weighting for Coding-Agent Routing
Nikhil Mahesh
(Independent, USA)
Before a coding agent can reason over a repository, it must decide which files to open. We isolate this pre-inference routing decision: given an issue report and a pre-fix tree, rank relevant paths without reading source code or invoking a model. On 407 frozen SWE-bench Verified instances, the deterministic cue-expanded pipeline achieved 25.3 percentage points higher Hit@20 than the original raw-path TF-IDF pipeline, a valid comparison of the pipelines as implemented. A factorized audit identified two important confounded implementation choices: asymmetric issue-path separator normalization and query-dependent TF-IDF fitting. With symmetric normalization and fixed candidate vectors, Hit@20 rises from 0.531 for raw words to 0.570 with each extracted cue added once and 0.595 with full cue weighting. The isolated full-cue gain is 0.064, with repository-cluster interval [0.022, 0.108]. In a target-string audit of the frozen cue-versus-plain pipelines, Hit@20 rises from 0.500 to 1.000 on 100 issues containing an exact gold path or standalone basename with extension, a 0.500 gain, and from 0.244 to 0.417 on the remaining 307 issues, a 0.173 gain. A prior-anchored character 3-10-gram baseline reaches 0.580. A fixed five-action cue-first controller replay then raises Hit@20 from 0.560 to 0.651 while lowering mean capped first-target-rank cost from 11.39 to 10.46; in an exploratory frozen 12-instance fixed-agent crossover, all three routes resolve 5/12 instances, and one run per route-instance cannot attribute the observed swaps to routing. The factorized analysis was conducted post hoc and requires prospective confirmation on unseen repositories; routing quality is not end-to-end repair quality. We contribute a controlled, reproducible account of how token alignment and deterministic cue weighting shape coding-agent routing, backed by complete rankings, agent receipts, and official harness outcomes.
Article Search
Artifacts Available
Article: asews26agenticdevmain-p117-p (type: Full Paper) doi:10.1145/3843282.3844439
AgenticDev '26: "An Evidence-Preserving Architecture ..."
An Evidence-Preserving Architecture for LLM-Assisted Vulnerability Review of Robotics Software
Anhtuan Le, Gregory Epiphaniou, and Carsten Maple
(University of Warwick, UK)
In industrial robotics, proprietary source often cannot leave the customer’s premises. Suppliers may therefore pair a static anal-yser with a large language model (LLM) running locally and send both outputs to one review queue. Combining the outputs can obscure two facts: which component supplied the evidence for a finding and which component may close it. We present an evidencepreserving architecture that keeps both facts visible. Each detector’s output remains a separate record, duplicates are linked rather than deleted, and every queue names the policy that derived it. A model’s negative verdict cannot deactivate another component’s finding. We instantiate the architecture for ROS 2 and C++ and evaluate it on 102 source windows. These include 40 vulnerable cases, each paired with a repaired twin. Seven local LLMs were run three times per case, and alternative merge, authority and invocation policies were replayed over the stored evidence. The results show that (1) conventional merging preserved coverage but erased most model attribution; (2) allowing a model verdict to close a case removed up to 15 confirmed issues; (3) the models rarely distinguished a defect from its repair; (4) retrieval added context but not accuracy because it could not reach a weakness class that the model had not proposed; and (5) 34% of model–issue detections were inconsistent across the three runs. Compact local LLMs can prioritise review and supply traceable evidence, but should not have authority to dismiss a security finding on their own.
Article Search
Article: asews26agenticdevmain-p99-p (type: Full Paper) doi:10.1145/3843282.3844435
AgenticDev '26: "Teach-to-Crash: A Closed-Loop ..."
Teach-to-Crash: A Closed-Loop Student–Teacher LLM Framework for Collision-Inducing Test Scenario Generation
Zaid Ghazal, Khouloud Gaaloul, and Bruce Maxim
(University of Michigan at Dearborn, USA)
Validating Autonomous Driving Systems (ADS) in simulation requires testing architectures that can discover rare, safety-critical failures while generating scenarios that are executable, diverse, and useful for downstream failure analysis. We introduce Teach-to-Crash, a closed-loop testing framework that combines a constrained ego-centric scenario representation, stagnation-aware search control, and a dual-LLM architecture for adaptive failure discovery. A high-reasoning Teacher LLM acts as an adaptive search controller, while a low-reasoning Student LLM emits simulator-executable scenarios in a strict JSON schema. The Teacher intervenes only when rolling collision rate and time-to-collision metrics stagnate, providing strategic guidance to redirect the search. In a CARLA case study with two experimental setups that vary the ego vehicle’s speed policy, Teach-to-Crash achieves the highest Collision Hit Rate (90.79%), the shortest mean Time-to-Collision (18.31s), and a competitive Collision Discovery Rate (136.21). PAFOT attains a higher mean CDR (179.44), but with substantially larger variance. Teach-to-Crash also yields the highest diversity (0.547) and, averaged across both setups on the CARLA Traffic Manager controller, the highest avoidability-based usefulness proxy (60.04%) among the compared methods. These results, within the evaluated CARLA scope, provide evidence that closed-loop dual-LLM reasoning can steer adversarial simulation-based testing over a constrained executable program space, generating failures that are frequent, structurally diverse, and assessed as more frequently avoidable.
Article Search
Artifacts Available
Article: asews26agenticdevmain-p126-p (type: Full Paper) doi:10.1145/3843282.3844441
AgenticDev '26: "Agents as Architecture Evaluators: ..."
Agents as Architecture Evaluators: Automating ATAM and SAAM for Continuous Assessment
Adyansh Kakran, Sathvika Miryala, and Karthik Vaidhyanathan
(IIIT Hyderabad, India)
The long-term success of a software system relies heavily on its un-derlying architecture, which dictates its performance with respect to quality attributes. Software Architecture Evaluation methods, such as the Architecture Tradeoff Analysis Method (ATAM) and the Software Architecture Analysis Method (SAAM), are widely recognized for their ability to mitigate risks and ensure reliable quality attributes. Their adoption remains limited due to cost, time, and expertise. We investigate agentic AI to lower these barriers. From human-conducted evaluations in the literature, we built a dataset of 39 cases and an agentic workflow that synthesizes evaluation reports from high-level architectural descriptions. Automated metrics and human evaluation yield relevance and completeness of 4.5/5 and 3.9/5. Agents automated scenario generation and risk identification, under 5 minutes at less than $0.50 per run.
Article Search
Article: asews26agenticdevmain-p78-p (type: Extended Abstract (2 pages)) doi:10.1145/3843282.3844431
AgenticDev '26: "Schema-Conditioned Next-Tool ..."
Schema-Conditioned Next-Tool Prediction for Coding Agents under Interface Drift
Selma Nyberg and Isak Samsten
(Stockholm University, Sweden)
Coding agents work in a loop that inspects the code, chooses a tool, observes the result, and repeats. Currently, every tool choice incurs a full LLM round trip. We study whether this control flow is predictable by a small model separate from the agent’s own LLM, and whether its predictability survives changes in how the tools are presented. We formulate next-tool prediction over the tool schemas available at inference time and study agent trajectories from three models on SWE-bench Lite under controlled perturbations to simulate interface drift. Our results show that a schema-conditioned LSTM reaches 0.848±0.005 top-1 accuracy on the unmodified interface and a mean of 0.805±0.002 across perturbations. We show that tool choice in coding agents is predictable, and that while a post-hoc aligned fixed-vocabulary classifier remains slightly more accurate (0.834 mean versus 0.805), schema conditioning achieves comparable robustness without requiring any alignment step, and degrades more uniformly under many-to-one drift.
Article Search
Article: asews26agenticdevmain-p75-p (type: Short Paper) doi:10.1145/3843282.3844430
AgenticDev '26: "Evidence-Driven, Failure-Resilient ..."
Evidence-Driven, Failure-Resilient Planning for Domain-Agnostic Multi-agent Orchestration
Baran Osmanoglu
(Saura AI, France)
LLM planners may misread requests, invent modules, omit steps, or fail when a model is unavailable. We treat planning reliability as a systems problem rather than primarily as a prompting problem. The control plane combines evidence collection, deterministic validation and repair, fallback handling, and human checkpoints to detect and recover from several failure classes. An ontology organizes roles and behavioural requirements as object-oriented classes and machine-checkable assertions. This supports domain extension while allowing valid module sequences to be checked against obligations rather than one exact plan. A manufacturing case demonstrates the separation of shared control from domain specific execution.
Article Search
Article: asews26agenticdevmain-p46-p (type: Extended Abstract (2 pages)) doi:10.1145/3843282.3844428
AgenticDev '26: "Execution-First Synthetic ..."
Execution-First Synthetic Tool-Use Trace Generation for LLM Agents
Hafsa Ouajdi, Francesco Giannuzzo, Alaa Boukhary, Paolo Papotti, Gerard Conangla, and Adam Elwood
(EURECOM, France; Aily Labs, Munich, Germany)
Agentic systems increasingly operate through executable workflows: they search artifacts, invoke tools, inspect structured observations, and query databases. Training such agents requires supervision that captures valid tool interactions, but query-first synthesis can produce plausible requests that do not correspond to valid tool sequences, compatible parameters, or available data. We propose SyntheticAgentTraceQA, an execution-first framework that constructs workflow templates, assigns tools under data-flow constraints, executes and validates traces, and only then synthesizes user tasks, teacher reasoning, and reference answers. Across the evaluated tool ecosystems, fine-tuning Qwen3.5 variants on the resulting data improves several measures of tool-use behavior, reference-trace agreement, and answer generation. Masked supervision, which excludes reasoning tokens from the loss, yields higher answer-production rates than full supervision and does not consistently improve trace agreement, especially at the 9B scale.
Article Search
Article: asews26agenticdevmain-p185-p (type: Extended Abstract (2 pages)) doi:10.1145/3843282.3844446
AgenticDev '26: "BDD-Bench: A Behavior-Driven ..."
BDD-Bench: A Behavior-Driven Benchmark for Coding Agents across the Application Lifecycle
Michał Jureczka, Anna Paleczek, Aiden Campbell, and Karol Przystalski
(Jagiellonian University, Poland; Exadel, Poland; AGH University of Krakow, Poland)
The main contribution of this paper is BDD-Bench, a repository-level benchmark in which coding agents implement executable Gherkin specifications across an application's development lifecycle. Its 111 chronologically ordered instances come from five Python repositories. In lifecycle mode, the agent solves consecutive instances on an evolving codebase, carrying each solution forward. Original tests are augmented with visible and hidden model-generated scenarios to strengthen behavioral coverage and assess generalization. Across thirteen models evaluated under a single agent harness with three runs each, the best-performing model, Claude Opus 4.8, resolves 45.0% of instances, covering 29.5% of the production code in the benchmark's reference patches when weighted by size. We further introduce a fine-grained failure taxonomy that pinpoints why lifecycle chains stall, and a matched-rerun design that isolates how agent-authored versus reference code histories shape an agent's downstream progress. These results show that even the strongest configurations often pass visible tests but fail withheld variations of the same behavioral specification.
Article Search
Article: asews26agenticdevmain-p25-p (type: Full Paper) doi:10.1145/3843282.3844426
AgenticDev '26: "Who Should Own the Loop? Harness ..."
Who Should Own the Loop? Harness Decomposition for Small-Model Repository Repair
Francesco Dente, Dario Satriani, Donatello Santoro, Enzo Veltri, and Paolo Papotti
(EURECOM, France; University of Basilicata, Italy)
Repository-level software repair is commonly approached by placing a language model in a free-form loop: the model explores the codebase, runs commands, edits files, interprets failures, and decides when the issue is resolved. This design works well for highly capable models, but it places many responsibilities on the model at once. When a smaller model fails, it is therefore difficult to tell whether it cannot produce the required patch, or whether it becomes lost while organizing the steps that lead to it. This paper studies an alternative design in which the repair process is decomposed into explicit stages. In particular, we separate finding the relevant code, constructing and using an executable signal of the bug, and generating the final edit. This perspective treats the harness not as a passive wrapper, but as part of the system being designed. We evaluate eight open-weight models from two families, ranging from 0.8B to 31B parameters, on a subset of SWE-bench Verified. Our results show that harness-managed control flow can substantially improve the effectiveness of the smallest models. For example, Gemma~4 E2B improves from 5.0% in the free-form loop to 27.4% in the decomposed repair loop and Gemma~4 E4B from 14.6% to 45.7%. Because the stages interact only through the context they build, they need not all be performed by the same model: handing localization and reproduction to stronger open-weight models, while the small model remains responsible only for the edit, raises the same two models further, to 39.8% and 53.7%. These findings suggest that small models can be useful for repository repair when responsibilities are divided across explicit stages that can be independently assigned to the component best suited to each.
Article Search
Article: asews26agenticdevmain-p131-p (type: Short Paper) doi:10.1145/3843282.3844442
AgenticDev '26: "Execution-First Synthetic ..."
Execution-First Synthetic Tool-Use Trace Generation for LLM Agents
Hafsa Ouajdi, Francesco Giannuzzo, Alaa Boukhary, Paolo Papotti, Gerard Conangla, and Adam Elwood
(EURECOM, France; Aily Labs, Munich, Germany)
Agentic systems increasingly operate through executable workflows: they search artifacts, invoke tools, inspect structured observations, and query databases. Training such agents requires supervision that captures valid tool interactions, but query-first synthesis can produce plausible requests that do not correspond to valid tool sequences, compatible parameters, or available data. We propose SyntheticAgentTraceQA, an execution-first framework that constructs workflow templates, assigns tools under data-flow constraints, executes and validates traces, and only then synthesizes user tasks, teacher reasoning, and reference answers. Across the evaluated tool ecosystems, fine-tuning Qwen3.5 variants on the resulting data improves several measures of tool-use behavior, reference-trace agreement, and answer generation. Masked supervision, which excludes reasoning tokens from the loss, yields higher answer-production rates than full supervision and does not consistently improve trace agreement, especially at the 9B scale.
Article Search
Article: asews26agenticdevmain-p185-p (type: Extended Abstract (2 pages)) doi:10.1145/3843282.3844446
AgenticDev '26: "Value-Preserving Architectures ..."
Value-Preserving Architectures for Agentic AI Systems
Alessandro Pesare, Tommaso Dolci, Katja Hose, and Emanuel Sallinger
(TU Wien, Austria)
The emergence of agentic AI and LLM-based multi-agent systems (MAS) presents unprecedented opportunities for automating complex tasks, while simultaneously raising critical concerns about the preservation of fundamental human-centered values, such as privacy, fairness, and safety. Although software engineering has traditionally focused on functional correctness, the adoption of LLMs and AI agents into complex socio-technical systems has intensified the need for responsible software engineering and robust value alignment. In MAS, architectural design decisions, such as coordination mechanisms, communication protocols, and system topologies, play a central role in shaping system behavior and the outcomes they produce. This paper argues that architectural choices influence not only the functionality and performance of MAS but can also promote value-oriented system behavior. Therefore, we investigate how different architectural designs support different human-centered values, discussing the following value-preserving architectural patterns: (i) a privacy-aware architecture with a federated topology, (ii) a distributed architecture to promote pluralism and diversity, and (iii) a guard-agent architecture to detect and mitigate unfairness. Finally, we introduce representative use cases to illustrate the proposed architectures in real-world scenarios. By linking architectural design with human-centered values, this work lays the foundation for a unified set of architectural patterns and guidelines towards the design of trustworthy MAS.
Article Search
Article: asews26agenticdevmain-p100-p (type: Short Paper) doi:10.1145/3843282.3844436
AgenticDev '26: "katIA: Hybrid Source Code ..."
katIA: Hybrid Source Code Change Impact Analysis
Katharina Stengg, Christian Macho, and Martin Pinzger
(University of Klagenfurt, Austria)
Maintaining software requires developers to identify the code entities affected by code changes. Existing approaches to change impact analysis (CIA) lack either high precision or high recall.Moreover, recent approaches are evaluated on benchmarks without labels for the relationships between changed code entities and therefore rely on simplified assumptions to determine these relationships. We propose katIA, a preliminary approach that combines static code analysis with LLM-based refinement for CIA. We implement two variants of katIA and evaluate them on a curated dataset. Our preliminary results suggest that both variants of katIA can improve CIA, with katIA-MA showing the strongest overall performance in our evaluation. These findings provide initial evidence that combining static code analysis and LLMs can benefit CIA.
Article Search
Article: asews26agenticdevmain-p2-p (type: Extended Abstract (2 pages)) doi:10.1145/3843282.3844420
AgenticDev '26: "MOSAICO: A Multi-agent Infrastructure ..."
MOSAICO: A Multi-agent Infrastructure for Collaborative LLM-Powered Software Engineering
Tuong Manh Vu, Ionut Predoaia, Konstantinos Barmpis, Dimitris Kolovos, and Antonio García-Domínguez
(University of York, UK)
Modern software engineering tasks increasingly call for Artificial Intelligence (AI) assistance that goes beyond a single large language model (LLM) acting alone. We present MOSAICO, an infrastructure for building and orchestrating communities of LLM-powered agents that collaborate to solve software engineering tasks. MOSAICO provides a reference client, a reference agent, and a set of task-specialised agent roles (solution, supervision, and consensus agents) that a collaboration agent invokes according to a workflow to complete a task. We showcase MOSAICO through three concrete demonstrations: a metamodelling use case, where a team of four agents jointly construct and validate a domain metamodel; a UML generation use case, where an agent converts a diagram image into textual notation; and a code documentation use case, where an agent uses client-side tools to generate documentation for existing code. Together, these demonstrations illustrate the generality of MOSAICO: a single, role-based agent infrastructure can support structurally different software engineering tasks with minimal reconfiguration.
Article Search
Article: asews26agenticdevmain-p159-p (type: Short Paper) doi:10.1145/3843282.3844444
AgenticDev '26: "Towards Model-Driven Generation ..."
Towards Model-Driven Generation of Domain-Specific Modelling Agents for Local Language Models
Antonio Garcia-Dominguez, Ionut Predoaia, Tuong Manh Vu, Konstantinos Barmpis, and Dimitris Kolovos
(University of York, UK)
Current frontier LLMs leverage vast training datasets with knowledge about a broad range of programming languages and frameworks, and can often solve tasks with minimal prompting. However, there are scenarios where remote LLMs are unsuitable, due to cost, privacy, security, or intellectual property concerns, and local language models need to be used. These smaller local language models typically require more careful guidance in order to achieve acceptable performance. In this paper, we evaluate the impact of using local models (in the 14-31B range) for object-oriented modelling, and we propose an approach for partially automating the creation of agents that "think in code" to solve such problems, instead of directly producing an output. We compare the direct invocation of Claude Sonnet 4.6 with that of several open-weight models, and with the use of those open-weight models while "thinking in code" as proposed. We observe that the best open-weight models, when "thinking in code", can produce models with similar Jaccard similarity scores to the model solutions in the Golden UML ModelSet dataset as Sonnet, although at the cost of increased token consumption. Based on the observed potential for generating domain-specific agents that reduce the demands on the LLM and therefore on the hardware requirements, we set out several lines of further work.
Article Search
Artifacts Available
Article: asews26agenticdevmain-p58-p (type: Full Paper) doi:10.1145/3843282.3844429
AgenticDev '26: "BDD-Bench: A Behavior-Driven ..."
BDD-Bench: A Behavior-Driven Benchmark for Coding Agents across the Application Lifecycle
Michał Jureczka, Anna Paleczek, Aiden Campbell, and Karol Przystalski
(Jagiellonian University, Poland; Exadel, Poland; AGH University of Krakow, Poland)
The main contribution of this paper is BDD-Bench, a repository-level benchmark in which coding agents implement executable Gherkin specifications across an application's development lifecycle. Its 111 chronologically ordered instances come from five Python repositories. In lifecycle mode, the agent solves consecutive instances on an evolving codebase, carrying each solution forward. Original tests are augmented with visible and hidden model-generated scenarios to strengthen behavioral coverage and assess generalization. Across thirteen models evaluated under a single agent harness with three runs each, the best-performing model, Claude Opus 4.8, resolves 45.0% of instances, covering 29.5% of the production code in the benchmark's reference patches when weighted by size. We further introduce a fine-grained failure taxonomy that pinpoints why lifecycle chains stall, and a matched-rerun design that isolates how agent-authored versus reference code histories shape an agent's downstream progress. These results show that even the strongest configurations often pass visible tests but fail withheld variations of the same behavioral specification.
Article Search
Article: asews26agenticdevmain-p25-p (type: Full Paper) doi:10.1145/3843282.3844426
AgenticDev '26: "Value-Preserving Architectures ..."
Value-Preserving Architectures for Agentic AI Systems
Alessandro Pesare, Tommaso Dolci, Katja Hose, and Emanuel Sallinger
(TU Wien, Austria)
The emergence of agentic AI and LLM-based multi-agent systems (MAS) presents unprecedented opportunities for automating complex tasks, while simultaneously raising critical concerns about the preservation of fundamental human-centered values, such as privacy, fairness, and safety. Although software engineering has traditionally focused on functional correctness, the adoption of LLMs and AI agents into complex socio-technical systems has intensified the need for responsible software engineering and robust value alignment. In MAS, architectural design decisions, such as coordination mechanisms, communication protocols, and system topologies, play a central role in shaping system behavior and the outcomes they produce. This paper argues that architectural choices influence not only the functionality and performance of MAS but can also promote value-oriented system behavior. Therefore, we investigate how different architectural designs support different human-centered values, discussing the following value-preserving architectural patterns: (i) a privacy-aware architecture with a federated topology, (ii) a distributed architecture to promote pluralism and diversity, and (iii) a guard-agent architecture to detect and mitigate unfairness. Finally, we introduce representative use cases to illustrate the proposed architectures in real-world scenarios. By linking architectural design with human-centered values, this work lays the foundation for a unified set of architectural patterns and guidelines towards the design of trustworthy MAS.
Article Search
Article: asews26agenticdevmain-p100-p (type: Short Paper) doi:10.1145/3843282.3844436
AgenticDev '26: "Schema-Conditioned Next-Tool ..."
Schema-Conditioned Next-Tool Prediction for Coding Agents under Interface Drift
Selma Nyberg and Isak Samsten
(Stockholm University, Sweden)
Coding agents work in a loop that inspects the code, chooses a tool, observes the result, and repeats. Currently, every tool choice incurs a full LLM round trip. We study whether this control flow is predictable by a small model separate from the agent’s own LLM, and whether its predictability survives changes in how the tools are presented. We formulate next-tool prediction over the tool schemas available at inference time and study agent trajectories from three models on SWE-bench Lite under controlled perturbations to simulate interface drift. Our results show that a schema-conditioned LSTM reaches 0.848±0.005 top-1 accuracy on the unmodified interface and a mean of 0.805±0.002 across perturbations. We show that tool choice in coding agents is predictable, and that while a post-hoc aligned fixed-vocabulary classifier remains slightly more accurate (0.834 mean versus 0.805), schema conditioning achieves comparable robustness without requiring any alignment step, and degrades more uniformly under many-to-one drift.
Article Search
Article: asews26agenticdevmain-p75-p (type: Short Paper) doi:10.1145/3843282.3844430
AgenticDev '26: "Who Should Own the Loop? Harness ..."
Who Should Own the Loop? Harness Decomposition for Small-Model Repository Repair
Francesco Dente, Dario Satriani, Donatello Santoro, Enzo Veltri, and Paolo Papotti
(EURECOM, France; University of Basilicata, Italy)
Repository-level software repair is commonly approached by placing a language model in a free-form loop: the model explores the codebase, runs commands, edits files, interprets failures, and decides when the issue is resolved. This design works well for highly capable models, but it places many responsibilities on the model at once. When a smaller model fails, it is therefore difficult to tell whether it cannot produce the required patch, or whether it becomes lost while organizing the steps that lead to it. This paper studies an alternative design in which the repair process is decomposed into explicit stages. In particular, we separate finding the relevant code, constructing and using an executable signal of the bug, and generating the final edit. This perspective treats the harness not as a passive wrapper, but as part of the system being designed. We evaluate eight open-weight models from two families, ranging from 0.8B to 31B parameters, on a subset of SWE-bench Verified. Our results show that harness-managed control flow can substantially improve the effectiveness of the smallest models. For example, Gemma~4 E2B improves from 5.0% in the free-form loop to 27.4% in the decomposed repair loop and Gemma~4 E4B from 14.6% to 45.7%. Because the stages interact only through the context they build, they need not all be performed by the same model: handing localization and reproduction to stronger open-weight models, while the small model remains responsible only for the edit, raises the same two models further, to 39.8% and 53.7%. These findings suggest that small models can be useful for repository repair when responsibilities are divided across explicit stages that can be independently assigned to the component best suited to each.
Article Search
Article: asews26agenticdevmain-p131-p (type: Short Paper) doi:10.1145/3843282.3844442
AgenticDev '26: "Automated Annotation of Test ..."
Automated Annotation of Test Types using LLMs: A Multi-LLM Workflow for Machine Learning Test-Related Commits
Asli Sari, Ayse Tosun, and Gulfem Isiklar Alptekin
(Istanbul Technical University, Türkiye; Galatasaray University, Türkiye)
Test-type annotation classifies commits based on the type of software test introduced or modified. While there are established taxonomies for testing machine learning (ML) software systems, implementing these manually at the commit level can be costly. This makes large language models (LLMs) a promising alternative. Utilizing an eight-category test-type taxonomy, we evaluate 97 manually labeled commits from scikit-learn using GPT-4o, Claude Sonnet, and Qwen 2.5-7B-Instruct-Turbo LLMs through four prompting strategies. This analysis focused on their classification performance, stability, agreement with experts, top-2 candidate recovery, and multi-LLM voting workflows (S1-S4) along with a corresponding analysis on Keras. Notably, GPT-4o in a zero-shot setting demonstrated the highest stability, and permitting a second label led to a reduction in single-prediction errors (e.g., Claude CoT-ZS Hit@1 = [59.79]%, Hit@2 = [77.32]%). Our most effective voting strategy (S1) surpassed the solo baseline, with its superior macro-F1 score (S1 > S2 > S3 > S4) also holding for Keras.
Article Search
Artifacts Available
Article: asews26agenticdevmain-p89-p (type: Extended Abstract (2 pages)) doi:10.1145/3843282.3844433
AgenticDev '26: "Who Should Own the Loop? Harness ..."
Who Should Own the Loop? Harness Decomposition for Small-Model Repository Repair
Francesco Dente, Dario Satriani, Donatello Santoro, Enzo Veltri, and Paolo Papotti
(EURECOM, France; University of Basilicata, Italy)
Repository-level software repair is commonly approached by placing a language model in a free-form loop: the model explores the codebase, runs commands, edits files, interprets failures, and decides when the issue is resolved. This design works well for highly capable models, but it places many responsibilities on the model at once. When a smaller model fails, it is therefore difficult to tell whether it cannot produce the required patch, or whether it becomes lost while organizing the steps that lead to it. This paper studies an alternative design in which the repair process is decomposed into explicit stages. In particular, we separate finding the relevant code, constructing and using an executable signal of the bug, and generating the final edit. This perspective treats the harness not as a passive wrapper, but as part of the system being designed. We evaluate eight open-weight models from two families, ranging from 0.8B to 31B parameters, on a subset of SWE-bench Verified. Our results show that harness-managed control flow can substantially improve the effectiveness of the smallest models. For example, Gemma~4 E2B improves from 5.0% in the free-form loop to 27.4% in the decomposed repair loop and Gemma~4 E4B from 14.6% to 45.7%. Because the stages interact only through the context they build, they need not all be performed by the same model: handing localization and reproduction to stronger open-weight models, while the small model remains responsible only for the edit, raises the same two models further, to 39.8% and 53.7%. These findings suggest that small models can be useful for repository repair when responsibilities are divided across explicit stages that can be independently assigned to the component best suited to each.
Article Search
Article: asews26agenticdevmain-p131-p (type: Short Paper) doi:10.1145/3843282.3844442
AgenticDev '26: "Bug Composition: Triggering ..."
Bug Composition: Triggering Multiple Bugs with Agentic Driver Generation
Vishva Arasan, Mahima Chaudhary, Wendy Contreras Martinez, Md Shafiuzzaman, Achintya Desai, and Tevfik Bultan
(University of California at Santa Barbara, USA)
Security vulnerabilities caused by bugs in software do not exist in isolation. Attackers can chain vulnerabilities caused by multiple bugs together to achieve their goals. However, determining whether multiple vulnerabilities can be triggered during a single execution is a difficult problem. In this paper, we present an agentic pipeline for automatically discovering whether a single shared input can trigger multiple bugs. In particular, we introduce an agentic approach that automatically generates predicates and constraint-based harnesses from buggy code, leveraging symbolic execution and SMT solving. We validate our approach using real-world vulnerabilities across three widely used, open source libraries: tcpdump, libxml2, and nDPI. Our pipeline successfully generates concrete shared inputs to trigger multiple bug pairs, including heap buffer overflows in tcpdump, stack and heap memory corruptions in libxml2 and out-of-bounds reads across independent protocol dissectors in nDPI. Furthermore, our methodology correctly identifies and proves the absence of shared inputs when bugs require mutually exclusive constraints. Our work introduces and demonstrates the feasibility of automated bug composition analysis, results of which can be valuable in identifying potential exploit chains and prioritizing critical vulnerabilities for patching.
Article Search
Article: asews26agenticdevmain-p97-p (type: Full Paper) doi:10.1145/3843282.3844434
AgenticDev '26: "Extender: A Multi-agent System ..."
Extender: A Multi-agent System for Code Extensibility Review
Jiahe Chen and Jiacheng Shen
(Duke Kunshan University, China)
Software systems must evolve as requirements and environments change, which makes extensibility a central quality goal. The SOLID principles, five object-oriented design principles, are widely endorsed in software engineering practice to evaluate software quality. However, automatically detecting violations of SOLID principles remains difficult since many violations are latent design flaws that only surface when the code must be extended. In this paper, we propose to check the violations of SOLID principles by explicitly extending and checking the extensibility of existing code. Based on this idea, we design , a multi-agent system that detects SOLID violations by generating principle-specific extension requirements, asking AI agents to implement them, and checking whether the resulting diffs expose design defects. Experimental results show that achieves 87.9% detection accuracy with a Qwen3.5 9B model, which is 32.1 points higher than the state-of-the-art LLM-based approaches.
Article Search
Article: asews26agenticdevmain-p21-p (type: Short Paper) doi:10.1145/3843282.3844424
AgenticDev '26: "Spec-Graph Meta-framework ..."
Spec-Graph Meta-framework for AI-Assisted Software Development: An Industry Case
Daniel Soto, Daniel González-Rueda, Mario Linares-Vásquez, and Andrés Hurtado
(Tres Astronautas, Colombia; Universidad de los Andes, Colombia; Blend 360, USA)
AI-assisted software development increasingly relies on large lan- guage models to transform natural-language specifications into code. However, in most current workflows, specifications are treated as inline prompt content, limiting their structure, reuse, and trace- ability across development tasks. This paper introduces a meta- framework for spec-driven AI-assisted development in which spec- ifications are represented as a typed, queryable graph consumed at runtime by software agents. The framework is organized around three principles: legibility, achieved through a typed hierarchy that distinguishes buildable artifacts from contextual constraints; addressability, enabled by a minimal set of edge types, substrate- agnostic graph materialization, and a cost-function-driven sequenc- ing mechanism; and longevity, maintained through continuous cu- ration of the specification substrate. We instantiate the framework in an industrial setting involving a multi-project software develop- ment organization with over fifty employees, using thirteen speci- fication types, a multi-substrate architecture, and four cooperating AI agents. To explore the proposed framework’s behavior in prac- tice we surveyed 19 practitioners in the organization after using the framework from one to five months.
Article Search
Article: asews26agenticdevmain-p86-p (type: Full Paper) doi:10.1145/3843282.3844432
AgenticDev '26: "Toward Automated Search for ..."
Toward Automated Search for Verifier-Spending Agentic Workflows: A Vision, Preliminary Evidence, and the Obstacles in the Way
Andrey Spiridonov
(AutoAW, Netherlands)
Agentic software-development systems increasingly wrap language models in workflows —repair loops, best-of-N samplers, multi-agent pipelines. We argue for a specific lens on why such structure helps and on how to design it automatically. Our position is that the value of an agentic workflow is the quality it converts a verifier into: a workflow is the control structure that turns a pass/fail signal (a test suite, a type checker, CI) into an edit, and a single monolithic model call—which emits an answer once and stops—structurally cannot spend a verifier at all. The design objective worth optimising is therefore verifier utilisation, and the natural tool is automated search over workflows that maximises it. We give preliminary evidence that verifier utilisation is the load-bearing variable (on a hard code benchmark, an executed test-repair loop moves a cheap model +14 points over an identical monolith), and we report, honestly, the obstacles we hit trying to make an automated search discover such workflows: agentic verifier-spending (a model choosing to call a test tool) is unreliable; prompting does not fix it; reliability requires treating verifier-spending as a deterministic operator. Once it is, a simple evolutionary search already selects the verifier-spending operator over verifier-blind alternatives; discovering such operators from scratch, and scaling the search, remain open. We frame the resulting research agenda.
Article Search
Article: asews26agenticdevmain-p134-p (type: Short Paper (5 pages)) doi:10.1145/3843282.3844443
AgenticDev '26: "katIA: Hybrid Source Code ..."
katIA: Hybrid Source Code Change Impact Analysis
Katharina Stengg, Christian Macho, and Martin Pinzger
(University of Klagenfurt, Austria)
Maintaining software requires developers to identify the code entities affected by code changes. Existing approaches to change impact analysis (CIA) lack either high precision or high recall.Moreover, recent approaches are evaluated on benchmarks without labels for the relationships between changed code entities and therefore rely on simplified assumptions to determine these relationships. We propose katIA, a preliminary approach that combines static code analysis with LLM-based refinement for CIA. We implement two variants of katIA and evaluate them on a curated dataset. Our preliminary results suggest that both variants of katIA can improve CIA, with katIA-MA showing the strongest overall performance in our evaluation. These findings provide initial evidence that combining static code analysis and LLMs can benefit CIA.
Article Search
Article: asews26agenticdevmain-p2-p (type: Extended Abstract (2 pages)) doi:10.1145/3843282.3844420
AgenticDev '26: "Evaluating Low-Shot Procedural ..."
Evaluating Low-Shot Procedural Skill Transfer in Language-Agent Debugging
Jiachen Sun
(ShanghaiTech University, China)
We study whether a natural-language SKILL.md induced from three debugging trajectories transfers to held-out language-agent debugging tasks. Our protocol records repair success, diagnosis-edit-test cycles, tokens, failed patches, and memory-attributed negative transfer. In a single-execution pilot on six verified PyBugHive black tasks, Auto SKILL.md ties a generic checklist and Reflexion at 5/6 solved. Task-level comparisons show lower process cost than reflection baselines but do not establish a stable advantage over the checklist. One negative-transfer case and an outlier-sensitive structural control illustrate why memory evaluation needs uncertainty, process, and misapplication measures rather than solve rate alone.
Article Search
Article: asews26agenticdevmain-p12-p (type: Extended Abstract (2 pages)) doi:10.1145/3843282.3844421
AgenticDev '26: "Automated Annotation of Test ..."
Automated Annotation of Test Types using LLMs: A Multi-LLM Workflow for Machine Learning Test-Related Commits
Asli Sari, Ayse Tosun, and Gulfem Isiklar Alptekin
(Istanbul Technical University, Türkiye; Galatasaray University, Türkiye)
Test-type annotation classifies commits based on the type of software test introduced or modified. While there are established taxonomies for testing machine learning (ML) software systems, implementing these manually at the commit level can be costly. This makes large language models (LLMs) a promising alternative. Utilizing an eight-category test-type taxonomy, we evaluate 97 manually labeled commits from scikit-learn using GPT-4o, Claude Sonnet, and Qwen 2.5-7B-Instruct-Turbo LLMs through four prompting strategies. This analysis focused on their classification performance, stability, agreement with experts, top-2 candidate recovery, and multi-LLM voting workflows (S1-S4) along with a corresponding analysis on Keras. Notably, GPT-4o in a zero-shot setting demonstrated the highest stability, and permitting a second label led to a reduction in single-prediction errors (e.g., Claude CoT-ZS Hit@1 = [59.79]%, Hit@2 = [77.32]%). Our most effective voting strategy (S1) surpassed the solo baseline, with its superior macro-F1 score (S1 > S2 > S3 > S4) also holding for Keras.
Article Search
Artifacts Available
Article: asews26agenticdevmain-p89-p (type: Extended Abstract (2 pages)) doi:10.1145/3843282.3844433
AgenticDev '26: "Agents as Architecture Evaluators: ..."
Agents as Architecture Evaluators: Automating ATAM and SAAM for Continuous Assessment
Adyansh Kakran, Sathvika Miryala, and Karthik Vaidhyanathan
(IIIT Hyderabad, India)
The long-term success of a software system relies heavily on its un-derlying architecture, which dictates its performance with respect to quality attributes. Software Architecture Evaluation methods, such as the Architecture Tradeoff Analysis Method (ATAM) and the Software Architecture Analysis Method (SAAM), are widely recognized for their ability to mitigate risks and ensure reliable quality attributes. Their adoption remains limited due to cost, time, and expertise. We investigate agentic AI to lower these barriers. From human-conducted evaluations in the literature, we built a dataset of 39 cases and an agentic workflow that synthesizes evaluation reports from high-level architectural descriptions. Automated metrics and human evaluation yield relevance and completeness of 4.5/5 and 3.9/5. Agents automated scenario generation and risk identification, under 5 minutes at less than $0.50 per run.
Article Search
Article: asews26agenticdevmain-p78-p (type: Extended Abstract (2 pages)) doi:10.1145/3843282.3844431
AgenticDev '26: "Who Should Own the Loop? Harness ..."
Who Should Own the Loop? Harness Decomposition for Small-Model Repository Repair
Francesco Dente, Dario Satriani, Donatello Santoro, Enzo Veltri, and Paolo Papotti
(EURECOM, France; University of Basilicata, Italy)
Repository-level software repair is commonly approached by placing a language model in a free-form loop: the model explores the codebase, runs commands, edits files, interprets failures, and decides when the issue is resolved. This design works well for highly capable models, but it places many responsibilities on the model at once. When a smaller model fails, it is therefore difficult to tell whether it cannot produce the required patch, or whether it becomes lost while organizing the steps that lead to it. This paper studies an alternative design in which the repair process is decomposed into explicit stages. In particular, we separate finding the relevant code, constructing and using an executable signal of the bug, and generating the final edit. This perspective treats the harness not as a passive wrapper, but as part of the system being designed. We evaluate eight open-weight models from two families, ranging from 0.8B to 31B parameters, on a subset of SWE-bench Verified. Our results show that harness-managed control flow can substantially improve the effectiveness of the smallest models. For example, Gemma~4 E2B improves from 5.0% in the free-form loop to 27.4% in the decomposed repair loop and Gemma~4 E4B from 14.6% to 45.7%. Because the stages interact only through the context they build, they need not all be performed by the same model: handing localization and reproduction to stronger open-weight models, while the small model remains responsible only for the edit, raises the same two models further, to 39.8% and 53.7%. These findings suggest that small models can be useful for repository repair when responsibilities are divided across explicit stages that can be independently assigned to the component best suited to each.
Article Search
Article: asews26agenticdevmain-p131-p (type: Short Paper) doi:10.1145/3843282.3844442
AgenticDev '26: "MOSAICO: A Multi-agent Infrastructure ..."
MOSAICO: A Multi-agent Infrastructure for Collaborative LLM-Powered Software Engineering
Tuong Manh Vu, Ionut Predoaia, Konstantinos Barmpis, Dimitris Kolovos, and Antonio García-Domínguez
(University of York, UK)
Modern software engineering tasks increasingly call for Artificial Intelligence (AI) assistance that goes beyond a single large language model (LLM) acting alone. We present MOSAICO, an infrastructure for building and orchestrating communities of LLM-powered agents that collaborate to solve software engineering tasks. MOSAICO provides a reference client, a reference agent, and a set of task-specialised agent roles (solution, supervision, and consensus agents) that a collaboration agent invokes according to a workflow to complete a task. We showcase MOSAICO through three concrete demonstrations: a metamodelling use case, where a team of four agents jointly construct and validate a domain metamodel; a UML generation use case, where an agent converts a diagram image into textual notation; and a code documentation use case, where an agent uses client-side tools to generate documentation for existing code. Together, these demonstrations illustrate the generality of MOSAICO: a single, role-based agent infrastructure can support structurally different software engineering tasks with minimal reconfiguration.
Article Search
Article: asews26agenticdevmain-p159-p (type: Short Paper) doi:10.1145/3843282.3844444
AgenticDev '26: "Towards Model-Driven Generation ..."
Towards Model-Driven Generation of Domain-Specific Modelling Agents for Local Language Models
Antonio Garcia-Dominguez, Ionut Predoaia, Tuong Manh Vu, Konstantinos Barmpis, and Dimitris Kolovos
(University of York, UK)
Current frontier LLMs leverage vast training datasets with knowledge about a broad range of programming languages and frameworks, and can often solve tasks with minimal prompting. However, there are scenarios where remote LLMs are unsuitable, due to cost, privacy, security, or intellectual property concerns, and local language models need to be used. These smaller local language models typically require more careful guidance in order to achieve acceptable performance. In this paper, we evaluate the impact of using local models (in the 14-31B range) for object-oriented modelling, and we propose an approach for partially automating the creation of agents that "think in code" to solve such problems, instead of directly producing an output. We compare the direct invocation of Claude Sonnet 4.6 with that of several open-weight models, and with the use of those open-weight models while "thinking in code" as proposed. We observe that the best open-weight models, when "thinking in code", can produce models with similar Jaccard similarity scores to the model solutions in the Golden UML ModelSet dataset as Sonnet, although at the cost of increased token consumption. Based on the observed potential for generating domain-specific agents that reduce the demands on the LLM and therefore on the hardware requirements, we set out several lines of further work.
Article Search
Artifacts Available
Article: asews26agenticdevmain-p58-p (type: Full Paper) doi:10.1145/3843282.3844429
AgenticDev '26: "Looping Is Not Reliability: ..."
Looping Is Not Reliability: State-Bound Evidence and Typed Revision Contracts for Agentic Code Repair
Xueping Gao, Jianwei Yang, and Qiang Yang
(Alibaba Cloud Computing, China)
Generate–test–revise loops are common in coding agents, but repetition alone provides no reliability guarantee. We study the gap between finding a correct patch and retaining, verifying, and submitting it. A sealed five-seed study over 30 HumanEval repairs produces 900 three-revision trajectories. Under forced revision, current correctness with current traces falls from 0.820 after one revision to 0.673 after two, although ever-correct rises to 0.847. Two common-state studies use 2,430 branches from identical frozen programs to remove post-treatment risk-set bias. In a prespecified 14B replication, stale traces harm 34/135 correct starts versus 4/135 with current traces, a 22.2-point increase (task-cluster 95% CI [8.9,37.0], exact Holm p=0.0337). A prospective 540-rollout policy eliminates observed correct-start harm but reduces wrong-start repair and fails its joint criterion. Repository experiments over 24 bugs and four coder stacks expose floor effects and component heterogeneity without Holm-significant effects. We therefore separate admission, preservation, grounded certification, competence, and liveness. We derive an evidence-bound typed loop contract and instantiate its mechanically enforceable subset in a reference implementation that binds verifier evidence to exact code states, preserves verified checkpoints, and emits auditable admission receipts. The implementation is an executable specification and conformance artifact, not evidence of improved repair competence or calibrated verifier dependence.
Article Search
Article: asews26agenticdevmain-p114-p (type: Full Paper) doi:10.1145/3843282.3844438
AgenticDev '26: "Looping Is Not Reliability: ..."
Looping Is Not Reliability: State-Bound Evidence and Typed Revision Contracts for Agentic Code Repair
Xueping Gao, Jianwei Yang, and Qiang Yang
(Alibaba Cloud Computing, China)
Generate–test–revise loops are common in coding agents, but repetition alone provides no reliability guarantee. We study the gap between finding a correct patch and retaining, verifying, and submitting it. A sealed five-seed study over 30 HumanEval repairs produces 900 three-revision trajectories. Under forced revision, current correctness with current traces falls from 0.820 after one revision to 0.673 after two, although ever-correct rises to 0.847. Two common-state studies use 2,430 branches from identical frozen programs to remove post-treatment risk-set bias. In a prespecified 14B replication, stale traces harm 34/135 correct starts versus 4/135 with current traces, a 22.2-point increase (task-cluster 95% CI [8.9,37.0], exact Holm p=0.0337). A prospective 540-rollout policy eliminates observed correct-start harm but reduces wrong-start repair and fails its joint criterion. Repository experiments over 24 bugs and four coder stacks expose floor effects and component heterogeneity without Holm-significant effects. We therefore separate admission, preservation, grounded certification, competence, and liveness. We derive an evidence-bound typed loop contract and instantiate its mechanically enforceable subset in a reference implementation that binds verifier evidence to exact code states, preserves verified checkpoints, and emits auditable admission receipts. The implementation is an executable specification and conformance artifact, not evidence of improved repair competence or calibrated verifier dependence.
Article Search
Article: asews26agenticdevmain-p114-p (type: Full Paper) doi:10.1145/3843282.3844438
AgenticDev '26: "Measuring the Wrong Number: ..."
Measuring the Wrong Number: Cost-Aware, Correctness-Gated Evaluation of Efficiency Claims in Multi-turn Agentic Coding
Erdni Zergetaev
(Independent Researcher, Kazakhstan)
A fast-growing ecosystem of plugins, skills, and prompt packs promises to make AI coding agents cheaper. The most popular single artifact—a "talk-like-a-caveman" skill for a widely used coding agent—has ~92,000 GitHub stars (as of 2026-07) and advertises a 65% token reduction. We show that such claims, though not fabricated, measure the wrong number. Efficiency tools are validated on single-shot question answering and report output-token reduction, whereas a real agentic coding session is dominated by input and cached context re-read every turn; the model's own output is only ~0.6% of session tokens in aggregate and ~20% of the dollar cost. We introduce a reproducible, cost-aware, correctness-gated A/B harness that installs each tool faithfully, runs repeated trials over multi-turn coding tasks, breaks the bill down by token class, and gates every trial on a structural correctness check. Across 140 runs on two widely used efficiency tools — the two most-starred in this category — neither the token claim nor the money behind it survives. Output falls far short of the advertised 63–75%, which an industry test had already found; the finding we add is that the dollar bill does not move at all, coming out statistically flat with confidence intervals that exclude any saving close to the advertised figures. The ceiling is also structural: output is only about a fifth of the bill, so even a perfect 65% output cut caps bill savings near ~13%. We argue that trustworthy evaluation of agent tooling must measure cost on realistic multi-turn sessions with a correctness gate, and we release the recorded measurements with a self-checking script that reproduces every number.
Article Search
Article: asews26agenticdevmain-p20-p (type: Short Paper) doi:10.1145/3843282.3844423