RASE '26: "NORMA: Automated AI Act Compliance ..."
NORMA: Automated AI Act Compliance Assessment for AI Training Datasets
Elisabetta Arba, Allegra Berardi, Bedilia Estrada-Torres, Francesco Leotta, Flavia Monti, and Massimo Mecella
(Sapienza University of Rome, Italy; University of Seville, Spain)
The EU Artificial Intelligence Act (AI Act, Regulation EU 2024/1689) imposes binding data quality requirements on training, validation, and test datasets for high-risk AI systems through Article 10, operationalised here as six dimensions: relevance, representativeness, completeness, freedom from errors, absence of bias, and governance. Yet the regulation specifies no quantitative metrics or practical tooling for verifying compliance, leaving practitioners with a significant operationalisation gap. We present NORMA, an interactive web application that translates Article 10's qualitative requirements into measurable thresholds grounded in the fairness and data-quality literature, integrates a large language model for context-sensitive interpretation, and produces an explainable, per-dimension compliance report tied to specific regulatory articles. We demonstrate NORMA on three real-world datasets from Annex III high-risk sectors, i.e., education, law enforcement, and credit assessment.
Article Search
Article: asews26rasemain-id23-p (type: Short Paper (4 pages)) doi:10.1145/3820756.3844914
RASE '26: "NORMA: Automated AI Act Compliance ..."
NORMA: Automated AI Act Compliance Assessment for AI Training Datasets
Elisabetta Arba, Allegra Berardi, Bedilia Estrada-Torres, Francesco Leotta, Flavia Monti, and Massimo Mecella
(Sapienza University of Rome, Italy; University of Seville, Spain)
The EU Artificial Intelligence Act (AI Act, Regulation EU 2024/1689) imposes binding data quality requirements on training, validation, and test datasets for high-risk AI systems through Article 10, operationalised here as six dimensions: relevance, representativeness, completeness, freedom from errors, absence of bias, and governance. Yet the regulation specifies no quantitative metrics or practical tooling for verifying compliance, leaving practitioners with a significant operationalisation gap. We present NORMA, an interactive web application that translates Article 10's qualitative requirements into measurable thresholds grounded in the fairness and data-quality literature, integrates a large language model for context-sensitive interpretation, and produces an explainable, per-dimension compliance report tied to specific regulatory articles. We demonstrate NORMA on three real-world datasets from Annex III high-risk sectors, i.e., education, law enforcement, and credit assessment.
Article Search
Article: asews26rasemain-id23-p (type: Short Paper (4 pages)) doi:10.1145/3820756.3844914
RASE '26: "TaxVerif: Verification-Oriented ..."
TaxVerif: Verification-Oriented COBOL Tax-Code Migration via Invariant Extraction and Semantic Validation
Vadim Tynchenko, Igor Masich, Dmitry Martysyuk, Yury Kostromin, Aleksei Borodulin, Ivan Malashin, Vladimir Nelyub, and Andrei Gantimurov
(Bauman Moscow State Technical University, Russia)
Migrating legacy COBOL tax software to modern programming languages requires preserving business semantics in addition to producing syntactically correct translations. Small deviations in thresholds, deduction rules, arithmetic expressions, or boundary conditions may lead to incorrect financial computations. This paper presents TaxVerif, a verification-oriented migration framework that combines invariant extraction, assertion-guided translation, and mutation-based semantic validation. Candidate invariants are extracted from COBOL programs, translated into executable assertions, and used to verify Java-like translations under bounded execution. The evaluation includes an invariant-extraction benchmark, a controlled synthetic mutation benchmark, and a real-code benchmark using open-source COBOL programs with lexical, structural, and graph-based features. The results show that invariant-guided verification substantially improves semantic-error detection over syntax-based validation, while strict evaluation on unseen programs and unseen mutation types remains significantly more challenging, emphasizing the need for semantic verification in legacy-code migration.
Article Search
Article: asews26rasemain-id7-p (type: Full Paper (8 pages)) doi:10.1145/3820756.3844907
RASE '26: "Software Engineering Governance: ..."
Software Engineering Governance: Foundations for Trustworthy Automated Software Engineering
Carlos Mario Braga, Manuel A. Serrano, and Eduardo Fernandez-Medina
(University of Castilla-La Mancha, Spain)
Automated software engineering is progressively evolving from supporting human engineers to actively performing engineering activities through AI assistants, autonomous agents, automated testing, vulnerability repair, and deployment pipelines. As automation becomes an active participant in the software lifecycle, trustworthiness concerns extend beyond software systems themselves to the engineering processes, organisational decisions, and automated actors involved in their development. This evolution elevates Software Engineering Governance from a supporting organisational activity to a central engineering concern.
The established challenge of translating normative trustworthiness principles into operational practice becomes particularly consequential when automated systems perform engineering activities directly. Software Engineering Governance provides an engineering perspective for connecting these domains by interpreting trustworthiness principles within organisational and regulatory contexts, establishing operational lifecycle artefacts that guide automated execution, and providing the basis for interpreting the operational evidence generated during execution in subsequent accountable evaluation.
Viewing trustworthy automation through this governance perspective reveals two complementary governance responsibilities. Before automated execution, trustworthiness objectives must be systematically operationalised into lifecycle artefacts that guide or constrain engineering behaviour. After execution, operational evidence must be interpreted to support the accountable evaluation of observed engineering behaviour. Their combination provides the organising perspective through which Software Engineering Governance relates governance decisions made before execution to the evidence available for evaluation afterwards
Article Search
Article: asews26rasemain-id29-p (type: Full Paper (8 pages)) doi:10.1145/3820756.3844918
RASE '26: "When Do Inference-Cost Stress ..."
When Do Inference-Cost Stress Tests Generalize? A Cross-Model and Cross-Configuration Measurement Study
Arman Chhetri and Yinxi Liu
(Rochester Institute of Technology, USA)
An inference-cost stressor is an input crafted to make a large language model (LLM) emit an abnormally long response, inflating the compute, latency, and energy that one request consumes. Automated stressor generators are therefore attractive as a source of regression-test inputs for resource robustness. A generator is itself an automated testing artifact, however, and its output is trustworthy as a test input only if it still saturates output on the exact model and deployment configuration under test—yet published generators are typically validated on a single model in a single configuration.
We measure how far that validation carries. Using the published Engorgio suffix generator on eight open-weight instruction-tuned models, a source-by-target reuse design, and system-prompt and quantization ablations, we ask when a generated stressor remains effective. We find that effectiveness is a property of the generator-target-configuration triple rather than of the stressor alone. Engorgio saturates output on four of the eight models but almost never on the other four, including the evaluated Llama and Qwen releases, where its suffix even makes responses shorter than an unmodified prompt. That negative result does not establish robustness: removing the chat template raises saturation on those same models to between 15% and 97% of trials, and a second generator (LoopLLM) reaches nonzero saturation on all four. Reuse across models is unreliable and is dominated by the target rather than by the model the suffix was optimized on, and susceptibility shifts sharply with the instruction-formatting layer while barely moving under quantization. Resource-robustness regression suites must therefore re-validate each stressor on the specific generator, target, and configuration under test. We release the trial data, reuse matrix, configuration templates, and analysis scripts.
Article Search
Artifacts Available
Article: asews26rasemain-id13-p (type: Full Paper (8 pages)) doi:10.1145/3820756.3844910
RASE '26: "When Log Parsing Removes What ..."
When Log Parsing Removes What Matters: Detecting Failures in Distributed GPU Workloads
Boyuan Cui, Yuyang Zeng, Yifang Tian, and Hans-Arno Jacobsen
(University of Toronto, Canada)
Large AI models increasingly rely on distributed GPU execution, yet it remains unclear whether established anomaly detection methods transfer effectively to this setting. We introduce GF-Bench, a PyTorch-based framework and dataset for generating controlled failures in distributed GPU workloads. The framework supports extensible workload and fault-injection logic; based on this design, the current release targets failures visible in multi-GPU communication logs and provides 220 labeled two-GPU executions covering normal runs, CUDA out-of-memory fail-stop failures, and persistent straggler fail-slow failures. Using GF-Bench, we evaluate three sequence-based methods (DeepLog, LogAnomaly, and LogBERT) and the LLM-based LogPrompt on parsed logs, and additionally evaluate LogPrompt on raw logs. On parsed inputs, no method reliably detects both failure classes, and the highest overall F1 score is 0.667. Raw inputs raise the overall F1 score of in-context Log- Prompt to 0.983, at 9.0× its average input-token usage and 8.7× its per-run cost. These results reveal a trade-off between effectiveness and efficiency: parsed inputs reduce LLM token use and inference cost but can discard the distributed execution context required for accurate failure detection. GF-Bench provides a reproducible basis for studying cost-efficient anomaly detection methods and can be extended to additional GPU workloads and failure types.
Article Search
Article: asews26rasemain-id32-p (type: Short Paper (4 pages)) doi:10.1145/3820756.3844919
RASE '26: "NORMA: Automated AI Act Compliance ..."
NORMA: Automated AI Act Compliance Assessment for AI Training Datasets
Elisabetta Arba, Allegra Berardi, Bedilia Estrada-Torres, Francesco Leotta, Flavia Monti, and Massimo Mecella
(Sapienza University of Rome, Italy; University of Seville, Spain)
The EU Artificial Intelligence Act (AI Act, Regulation EU 2024/1689) imposes binding data quality requirements on training, validation, and test datasets for high-risk AI systems through Article 10, operationalised here as six dimensions: relevance, representativeness, completeness, freedom from errors, absence of bias, and governance. Yet the regulation specifies no quantitative metrics or practical tooling for verifying compliance, leaving practitioners with a significant operationalisation gap. We present NORMA, an interactive web application that translates Article 10's qualitative requirements into measurable thresholds grounded in the fairness and data-quality literature, integrates a large language model for context-sensitive interpretation, and produces an explainable, per-dimension compliance report tied to specific regulatory articles. We demonstrate NORMA on three real-world datasets from Annex III high-risk sectors, i.e., education, law enforcement, and credit assessment.
Article Search
Article: asews26rasemain-id23-p (type: Short Paper (4 pages)) doi:10.1145/3820756.3844914
RASE '26: "Software Engineering Governance: ..."
Software Engineering Governance: Foundations for Trustworthy Automated Software Engineering
Carlos Mario Braga, Manuel A. Serrano, and Eduardo Fernandez-Medina
(University of Castilla-La Mancha, Spain)
Automated software engineering is progressively evolving from supporting human engineers to actively performing engineering activities through AI assistants, autonomous agents, automated testing, vulnerability repair, and deployment pipelines. As automation becomes an active participant in the software lifecycle, trustworthiness concerns extend beyond software systems themselves to the engineering processes, organisational decisions, and automated actors involved in their development. This evolution elevates Software Engineering Governance from a supporting organisational activity to a central engineering concern.
The established challenge of translating normative trustworthiness principles into operational practice becomes particularly consequential when automated systems perform engineering activities directly. Software Engineering Governance provides an engineering perspective for connecting these domains by interpreting trustworthiness principles within organisational and regulatory contexts, establishing operational lifecycle artefacts that guide automated execution, and providing the basis for interpreting the operational evidence generated during execution in subsequent accountable evaluation.
Viewing trustworthy automation through this governance perspective reveals two complementary governance responsibilities. Before automated execution, trustworthiness objectives must be systematically operationalised into lifecycle artefacts that guide or constrain engineering behaviour. After execution, operational evidence must be interpreted to support the accountable evaluation of observed engineering behaviour. Their combination provides the organising perspective through which Software Engineering Governance relates governance decisions made before execution to the evidence available for evaluation afterwards
Article Search
Article: asews26rasemain-id29-p (type: Full Paper (8 pages)) doi:10.1145/3820756.3844918
RASE '26: "TaxVerif: Verification-Oriented ..."
TaxVerif: Verification-Oriented COBOL Tax-Code Migration via Invariant Extraction and Semantic Validation
Vadim Tynchenko, Igor Masich, Dmitry Martysyuk, Yury Kostromin, Aleksei Borodulin, Ivan Malashin, Vladimir Nelyub, and Andrei Gantimurov
(Bauman Moscow State Technical University, Russia)
Migrating legacy COBOL tax software to modern programming languages requires preserving business semantics in addition to producing syntactically correct translations. Small deviations in thresholds, deduction rules, arithmetic expressions, or boundary conditions may lead to incorrect financial computations. This paper presents TaxVerif, a verification-oriented migration framework that combines invariant extraction, assertion-guided translation, and mutation-based semantic validation. Candidate invariants are extracted from COBOL programs, translated into executable assertions, and used to verify Java-like translations under bounded execution. The evaluation includes an invariant-extraction benchmark, a controlled synthetic mutation benchmark, and a real-code benchmark using open-source COBOL programs with lexical, structural, and graph-based features. The results show that invariant-guided verification substantially improves semantic-error detection over syntax-based validation, while strict evaluation on unseen programs and unseen mutation types remains significantly more challenging, emphasizing the need for semantic verification in legacy-code migration.
Article Search
Article: asews26rasemain-id7-p (type: Full Paper (8 pages)) doi:10.1145/3820756.3844907
RASE '26: "Minimizing Vulnerability Prevalence ..."
Minimizing Vulnerability Prevalence in Maven Dependencies via Answer Set Programming
Christian Macho, Laura Hofer, and Martin Pinzger
(University of Klagenfurt, Austria)
Software projects rely extensively on third-party libraries, but dependency versions with known vulnerabilities can expose them to security risks. Updating these dependencies is challenging because alternative versions may introduce different transitive dependencies, additional vulnerabilities, or compatibility failures. Moreover, not every dependency for which a vulnerability is reported is necessarily used by the project.
We present an approach that uses Answer Set Programming to identify less vulnerable dependency configurations for Apache Maven projects. The approach combines vulnerability information from three security advisories with the resolved dependency tree, alternative dependency subtrees, and the project's callgraph which identifies dependencies reached from the project. Clingo, our solver, then outputs ranked dependency configurations, reducing the number of vulnerabilities while maintaining buildability.
We evaluated our approach on twelve open-source Java projects and found that only ten of them actually use dependencies with vulnerabilities. For seven of the remaining ten projects, our approach identifies a buildable configuration with fewer vulnerabilities, reducing their number by 40.3% and their weighted vulnerability score by 41.4%. We also show that our approach is feasible in practice, yielding a median runtime of 15 and 2 minutes overall and for the optimization step only, respectively.
Article Search
Article: asews26rasemain-id27-p (type: Full Paper (8 pages)) doi:10.1145/3820756.3844916
RASE '26: "Whose Specification Did You ..."
Whose Specification Did You Test? The Hidden Interpretation Bias in LLM-Generated Test Suites
Md Mainul Islam
(United International University, Dhaka, Bangladesh)
LLMs are increasingly used to generate both software implementations and the test suites used to evaluate them. This raises a fundamental question: when an API specification leaves behaviour under-specified, do LLM-generated tests act as neutral observers of correctness, or do they encode interpretation choices that favour implementations sharing those choices? We investigate this question through a controlled experiment in which identical LLM-generated test suites are applied to independently developed human-coded and LLM-coded REST API implementations. We examine three test-generation strategies that vary in coverage and emphasis on ambiguous behaviours. The results provide statistically significant evidence of interpretation-coupled testing: test outcomes depend partly on agreement between the test suite's implicit interpretation and the implementation's interpretation of the specification. We discuss the implications for autonomous software engineering pipelines in which a single model generates both code and tests from the same specification, and identify practical directions for interpretation-aware test design.
Article Search
Artifacts Available
Article: asews26rasemain-id8-p (type: Short Paper (4 pages)) doi:10.1145/3820756.3844908
RASE '26: "When Log Parsing Removes What ..."
When Log Parsing Removes What Matters: Detecting Failures in Distributed GPU Workloads
Boyuan Cui, Yuyang Zeng, Yifang Tian, and Hans-Arno Jacobsen
(University of Toronto, Canada)
Large AI models increasingly rely on distributed GPU execution, yet it remains unclear whether established anomaly detection methods transfer effectively to this setting. We introduce GF-Bench, a PyTorch-based framework and dataset for generating controlled failures in distributed GPU workloads. The framework supports extensible workload and fault-injection logic; based on this design, the current release targets failures visible in multi-GPU communication logs and provides 220 labeled two-GPU executions covering normal runs, CUDA out-of-memory fail-stop failures, and persistent straggler fail-slow failures. Using GF-Bench, we evaluate three sequence-based methods (DeepLog, LogAnomaly, and LogBERT) and the LLM-based LogPrompt on parsed logs, and additionally evaluate LogPrompt on raw logs. On parsed inputs, no method reliably detects both failure classes, and the highest overall F1 score is 0.667. Raw inputs raise the overall F1 score of in-context Log- Prompt to 0.983, at 9.0× its average input-token usage and 8.7× its per-run cost. These results reveal a trade-off between effectiveness and efficiency: parsed inputs reduce LLM token use and inference cost but can discard the distributed execution context required for accurate failure detection. GF-Bench provides a reproducible basis for studying cost-efficient anomaly detection methods and can be extended to additional GPU workloads and failure types.
Article Search
Article: asews26rasemain-id32-p (type: Short Paper (4 pages)) doi:10.1145/3820756.3844919
RASE '26: "How Grammar-Based Synthetic ..."
How Grammar-Based Synthetic Data Shapes Behavior Learning: An Empirical Study of Input Generation Strategies
Tural Mammadov, Alireza Kheradmand, Marius Smytzek, and Andreas Zeller
(CISPA Helmholtz Center for Information Security, Germany; Saarland University, Germany)
Grammar-based fuzzing can synthesize valid inputs for programs that process structured data. For behavior learning, the same inputs can be paired with the outputs a program produces, creating datasets that train neural program mocks. The fuzzer then serves as a data generator, and its selection policy determines which behaviors the model observes during training. To study how grammar-based input generation affects neural behavior modeling, we compare four generator configurations, train encoder-decoder Transformer models on 10,000 and 100,000 generated input-output pairs, and evaluate the resulting models on synthetic hold-out data and available real-world samples. On synthetic hold-out data, the larger training corpus improves the BLEU score in all 48 paired comparisons; random generation and k-path guidance are strongest on average, though the best generator remains subject-dependent. On real-world data, the larger corpus improves the BLEU score in 35 of 40 paired comparisons, but the gains are smaller and less consistent. JSON-to-YAML models retain high real-world accuracy, whereas msticpy and py-asciimath models expose large synthetic-to-real quality gaps. Overall, generator selection affects learned behavior models, but practical performance depends strongly on how well grammar-generated inputs match the deployment distribution.
Article Search
Article: asews26rasemain-id21-p (type: Full Paper (8 pages)) doi:10.1145/3820756.3844912
RASE '26: "Can We Trust Strong Teachers? ..."
Can We Trust Strong Teachers? Auditing Knowledge Distillation for Binary Code Similarity Detection
Minh Khang Van, Yunru Wang, and Johannes Kinder
(LMU Munich, Germany)
Large binary models are costly to use for Binary Code Similarity Detection (BCSD) at scale, motivating their distillation into compact encoders. Existing BCSD distillation work focuses mainly on teacher reliability, while student-side factors remain underexplored. We conduct a controlled study on a large-scale cross-project benchmark, fixing the student architecture and task objective while comparing three distillation objectives with the retrieval-specialized CLAP and the best-performing objective with the generative decompiler Nova. Our results show that direct task training produces a strong student that nearly matches zero-shot CLAP, while distillation adds only marginal gains. We therefore study two student-side factors that shape how teacher supervision is received: its interaction with task training and the input visible to the student. Under the same loss coefficient, different distillation objectives produce markedly different gradient interactions with InfoNCE, yet these statistics do not yield a simple rule for loss balancing. Increasing the student context from 64 to 1,024 tokens consistently improves embedding distillation and narrows the gap to CLAP. These results show that reliable BCSD distillation depends not only on teacher quality, but also on the student's optimization and input constraints.
Article Search
Article: asews26rasemain-id28-p (type: Full Paper (Replication Study 8 pages)) doi:10.1145/3820756.3844917
RASE '26: "Toward Defensible System Behavior ..."
Toward Defensible System Behavior for AI-Generated Software Artifacts
Radosław Klimek
(AGH University of Krakow, Poland)
AI-generated software artifacts often lack explicit design rationale and globally consistent behavioural assumptions, as independently generated fragments may embed locally plausible but mutually inconsistent assumptions. Existing verification techniques provide strong guarantees when relevant properties are explicitly specified, but may not capture undocumented assumptions or behavioural conflicts emerging only after artifact composition. This vision paper argues that trustworthy AI-assisted software engineering requires behavioural accountability: making implicit assumptions explicit, surfacing behavioural violations, and producing auditable evidence. We outline a conceptual deductive layer over behavioural abstractions of AI-generated artifacts, complementing testing and verification with structured accountability evidence for developers and CI pipelines.
Article Search
Article: asews26rasemain-id14-p (type: Short Paper (4 pages)) doi:10.1145/3820756.3844911
RASE '26: "TaxVerif: Verification-Oriented ..."
TaxVerif: Verification-Oriented COBOL Tax-Code Migration via Invariant Extraction and Semantic Validation
Vadim Tynchenko, Igor Masich, Dmitry Martysyuk, Yury Kostromin, Aleksei Borodulin, Ivan Malashin, Vladimir Nelyub, and Andrei Gantimurov
(Bauman Moscow State Technical University, Russia)
Migrating legacy COBOL tax software to modern programming languages requires preserving business semantics in addition to producing syntactically correct translations. Small deviations in thresholds, deduction rules, arithmetic expressions, or boundary conditions may lead to incorrect financial computations. This paper presents TaxVerif, a verification-oriented migration framework that combines invariant extraction, assertion-guided translation, and mutation-based semantic validation. Candidate invariants are extracted from COBOL programs, translated into executable assertions, and used to verify Java-like translations under bounded execution. The evaluation includes an invariant-extraction benchmark, a controlled synthetic mutation benchmark, and a real-code benchmark using open-source COBOL programs with lexical, structural, and graph-based features. The results show that invariant-guided verification substantially improves semantic-error detection over syntax-based validation, while strict evaluation on unseen programs and unseen mutation types remains significantly more challenging, emphasizing the need for semantic verification in legacy-code migration.
Article Search
Article: asews26rasemain-id7-p (type: Full Paper (8 pages)) doi:10.1145/3820756.3844907
RASE '26: "NORMA: Automated AI Act Compliance ..."
NORMA: Automated AI Act Compliance Assessment for AI Training Datasets
Elisabetta Arba, Allegra Berardi, Bedilia Estrada-Torres, Francesco Leotta, Flavia Monti, and Massimo Mecella
(Sapienza University of Rome, Italy; University of Seville, Spain)
The EU Artificial Intelligence Act (AI Act, Regulation EU 2024/1689) imposes binding data quality requirements on training, validation, and test datasets for high-risk AI systems through Article 10, operationalised here as six dimensions: relevance, representativeness, completeness, freedom from errors, absence of bias, and governance. Yet the regulation specifies no quantitative metrics or practical tooling for verifying compliance, leaving practitioners with a significant operationalisation gap. We present NORMA, an interactive web application that translates Article 10's qualitative requirements into measurable thresholds grounded in the fairness and data-quality literature, integrates a large language model for context-sensitive interpretation, and produces an explainable, per-dimension compliance report tied to specific regulatory articles. We demonstrate NORMA on three real-world datasets from Annex III high-risk sectors, i.e., education, law enforcement, and credit assessment.
Article Search
Article: asews26rasemain-id23-p (type: Short Paper (4 pages)) doi:10.1145/3820756.3844914
RASE '26: "When Do Inference-Cost Stress ..."
When Do Inference-Cost Stress Tests Generalize? A Cross-Model and Cross-Configuration Measurement Study
Arman Chhetri and Yinxi Liu
(Rochester Institute of Technology, USA)
An inference-cost stressor is an input crafted to make a large language model (LLM) emit an abnormally long response, inflating the compute, latency, and energy that one request consumes. Automated stressor generators are therefore attractive as a source of regression-test inputs for resource robustness. A generator is itself an automated testing artifact, however, and its output is trustworthy as a test input only if it still saturates output on the exact model and deployment configuration under test—yet published generators are typically validated on a single model in a single configuration.
We measure how far that validation carries. Using the published Engorgio suffix generator on eight open-weight instruction-tuned models, a source-by-target reuse design, and system-prompt and quantization ablations, we ask when a generated stressor remains effective. We find that effectiveness is a property of the generator-target-configuration triple rather than of the stressor alone. Engorgio saturates output on four of the eight models but almost never on the other four, including the evaluated Llama and Qwen releases, where its suffix even makes responses shorter than an unmodified prompt. That negative result does not establish robustness: removing the chat template raises saturation on those same models to between 15% and 97% of trials, and a second generator (LoopLLM) reaches nonzero saturation on all four. Reuse across models is unreliable and is dominated by the target rather than by the model the suffix was optimized on, and susceptibility shifts sharply with the instruction-formatting layer while barely moving under quantization. Resource-robustness regression suites must therefore re-validate each stressor on the specific generator, target, and configuration under test. We release the trial data, reuse matrix, configuration templates, and analysis scripts.
Article Search
Artifacts Available
Article: asews26rasemain-id13-p (type: Full Paper (8 pages)) doi:10.1145/3820756.3844910
RASE '26: "Minimizing Vulnerability Prevalence ..."
Minimizing Vulnerability Prevalence in Maven Dependencies via Answer Set Programming
Christian Macho, Laura Hofer, and Martin Pinzger
(University of Klagenfurt, Austria)
Software projects rely extensively on third-party libraries, but dependency versions with known vulnerabilities can expose them to security risks. Updating these dependencies is challenging because alternative versions may introduce different transitive dependencies, additional vulnerabilities, or compatibility failures. Moreover, not every dependency for which a vulnerability is reported is necessarily used by the project.
We present an approach that uses Answer Set Programming to identify less vulnerable dependency configurations for Apache Maven projects. The approach combines vulnerability information from three security advisories with the resolved dependency tree, alternative dependency subtrees, and the project's callgraph which identifies dependencies reached from the project. Clingo, our solver, then outputs ranked dependency configurations, reducing the number of vulnerabilities while maintaining buildability.
We evaluated our approach on twelve open-source Java projects and found that only ten of them actually use dependencies with vulnerabilities. For seven of the remaining ten projects, our approach identifies a buildable configuration with fewer vulnerabilities, reducing their number by 40.3% and their weighted vulnerability score by 41.4%. We also show that our approach is feasible in practice, yielding a median runtime of 15 and 2 minutes overall and for the optimization step only, respectively.
Article Search
Article: asews26rasemain-id27-p (type: Full Paper (8 pages)) doi:10.1145/3820756.3844916
RASE '26: "TaxVerif: Verification-Oriented ..."
TaxVerif: Verification-Oriented COBOL Tax-Code Migration via Invariant Extraction and Semantic Validation
Vadim Tynchenko, Igor Masich, Dmitry Martysyuk, Yury Kostromin, Aleksei Borodulin, Ivan Malashin, Vladimir Nelyub, and Andrei Gantimurov
(Bauman Moscow State Technical University, Russia)
Migrating legacy COBOL tax software to modern programming languages requires preserving business semantics in addition to producing syntactically correct translations. Small deviations in thresholds, deduction rules, arithmetic expressions, or boundary conditions may lead to incorrect financial computations. This paper presents TaxVerif, a verification-oriented migration framework that combines invariant extraction, assertion-guided translation, and mutation-based semantic validation. Candidate invariants are extracted from COBOL programs, translated into executable assertions, and used to verify Java-like translations under bounded execution. The evaluation includes an invariant-extraction benchmark, a controlled synthetic mutation benchmark, and a real-code benchmark using open-source COBOL programs with lexical, structural, and graph-based features. The results show that invariant-guided verification substantially improves semantic-error detection over syntax-based validation, while strict evaluation on unseen programs and unseen mutation types remains significantly more challenging, emphasizing the need for semantic verification in legacy-code migration.
Article Search
Article: asews26rasemain-id7-p (type: Full Paper (8 pages)) doi:10.1145/3820756.3844907
RASE '26: "How Grammar-Based Synthetic ..."
How Grammar-Based Synthetic Data Shapes Behavior Learning: An Empirical Study of Input Generation Strategies
Tural Mammadov, Alireza Kheradmand, Marius Smytzek, and Andreas Zeller
(CISPA Helmholtz Center for Information Security, Germany; Saarland University, Germany)
Grammar-based fuzzing can synthesize valid inputs for programs that process structured data. For behavior learning, the same inputs can be paired with the outputs a program produces, creating datasets that train neural program mocks. The fuzzer then serves as a data generator, and its selection policy determines which behaviors the model observes during training. To study how grammar-based input generation affects neural behavior modeling, we compare four generator configurations, train encoder-decoder Transformer models on 10,000 and 100,000 generated input-output pairs, and evaluate the resulting models on synthetic hold-out data and available real-world samples. On synthetic hold-out data, the larger training corpus improves the BLEU score in all 48 paired comparisons; random generation and k-path guidance are strongest on average, though the best generator remains subject-dependent. On real-world data, the larger corpus improves the BLEU score in 35 of 40 paired comparisons, but the gains are smaller and less consistent. JSON-to-YAML models retain high real-world accuracy, whereas msticpy and py-asciimath models expose large synthetic-to-real quality gaps. Overall, generator selection affects learned behavior models, but practical performance depends strongly on how well grammar-generated inputs match the deployment distribution.
Article Search
Article: asews26rasemain-id21-p (type: Full Paper (8 pages)) doi:10.1145/3820756.3844912
RASE '26: "TaxVerif: Verification-Oriented ..."
TaxVerif: Verification-Oriented COBOL Tax-Code Migration via Invariant Extraction and Semantic Validation
Vadim Tynchenko, Igor Masich, Dmitry Martysyuk, Yury Kostromin, Aleksei Borodulin, Ivan Malashin, Vladimir Nelyub, and Andrei Gantimurov
(Bauman Moscow State Technical University, Russia)
Migrating legacy COBOL tax software to modern programming languages requires preserving business semantics in addition to producing syntactically correct translations. Small deviations in thresholds, deduction rules, arithmetic expressions, or boundary conditions may lead to incorrect financial computations. This paper presents TaxVerif, a verification-oriented migration framework that combines invariant extraction, assertion-guided translation, and mutation-based semantic validation. Candidate invariants are extracted from COBOL programs, translated into executable assertions, and used to verify Java-like translations under bounded execution. The evaluation includes an invariant-extraction benchmark, a controlled synthetic mutation benchmark, and a real-code benchmark using open-source COBOL programs with lexical, structural, and graph-based features. The results show that invariant-guided verification substantially improves semantic-error detection over syntax-based validation, while strict evaluation on unseen programs and unseen mutation types remains significantly more challenging, emphasizing the need for semantic verification in legacy-code migration.
Article Search
Article: asews26rasemain-id7-p (type: Full Paper (8 pages)) doi:10.1145/3820756.3844907
RASE '26: "TaxVerif: Verification-Oriented ..."
TaxVerif: Verification-Oriented COBOL Tax-Code Migration via Invariant Extraction and Semantic Validation
Vadim Tynchenko, Igor Masich, Dmitry Martysyuk, Yury Kostromin, Aleksei Borodulin, Ivan Malashin, Vladimir Nelyub, and Andrei Gantimurov
(Bauman Moscow State Technical University, Russia)
Migrating legacy COBOL tax software to modern programming languages requires preserving business semantics in addition to producing syntactically correct translations. Small deviations in thresholds, deduction rules, arithmetic expressions, or boundary conditions may lead to incorrect financial computations. This paper presents TaxVerif, a verification-oriented migration framework that combines invariant extraction, assertion-guided translation, and mutation-based semantic validation. Candidate invariants are extracted from COBOL programs, translated into executable assertions, and used to verify Java-like translations under bounded execution. The evaluation includes an invariant-extraction benchmark, a controlled synthetic mutation benchmark, and a real-code benchmark using open-source COBOL programs with lexical, structural, and graph-based features. The results show that invariant-guided verification substantially improves semantic-error detection over syntax-based validation, while strict evaluation on unseen programs and unseen mutation types remains significantly more challenging, emphasizing the need for semantic verification in legacy-code migration.
Article Search
Article: asews26rasemain-id7-p (type: Full Paper (8 pages)) doi:10.1145/3820756.3844907
RASE '26: "NORMA: Automated AI Act Compliance ..."
NORMA: Automated AI Act Compliance Assessment for AI Training Datasets
Elisabetta Arba, Allegra Berardi, Bedilia Estrada-Torres, Francesco Leotta, Flavia Monti, and Massimo Mecella
(Sapienza University of Rome, Italy; University of Seville, Spain)
The EU Artificial Intelligence Act (AI Act, Regulation EU 2024/1689) imposes binding data quality requirements on training, validation, and test datasets for high-risk AI systems through Article 10, operationalised here as six dimensions: relevance, representativeness, completeness, freedom from errors, absence of bias, and governance. Yet the regulation specifies no quantitative metrics or practical tooling for verifying compliance, leaving practitioners with a significant operationalisation gap. We present NORMA, an interactive web application that translates Article 10's qualitative requirements into measurable thresholds grounded in the fairness and data-quality literature, integrates a large language model for context-sensitive interpretation, and produces an explainable, per-dimension compliance report tied to specific regulatory articles. We demonstrate NORMA on three real-world datasets from Annex III high-risk sectors, i.e., education, law enforcement, and credit assessment.
Article Search
Article: asews26rasemain-id23-p (type: Short Paper (4 pages)) doi:10.1145/3820756.3844914
RASE '26: "NORMA: Automated AI Act Compliance ..."
NORMA: Automated AI Act Compliance Assessment for AI Training Datasets
Elisabetta Arba, Allegra Berardi, Bedilia Estrada-Torres, Francesco Leotta, Flavia Monti, and Massimo Mecella
(Sapienza University of Rome, Italy; University of Seville, Spain)
The EU Artificial Intelligence Act (AI Act, Regulation EU 2024/1689) imposes binding data quality requirements on training, validation, and test datasets for high-risk AI systems through Article 10, operationalised here as six dimensions: relevance, representativeness, completeness, freedom from errors, absence of bias, and governance. Yet the regulation specifies no quantitative metrics or practical tooling for verifying compliance, leaving practitioners with a significant operationalisation gap. We present NORMA, an interactive web application that translates Article 10's qualitative requirements into measurable thresholds grounded in the fairness and data-quality literature, integrates a large language model for context-sensitive interpretation, and produces an explainable, per-dimension compliance report tied to specific regulatory articles. We demonstrate NORMA on three real-world datasets from Annex III high-risk sectors, i.e., education, law enforcement, and credit assessment.
Article Search
Article: asews26rasemain-id23-p (type: Short Paper (4 pages)) doi:10.1145/3820756.3844914
RASE '26: "TaxVerif: Verification-Oriented ..."
TaxVerif: Verification-Oriented COBOL Tax-Code Migration via Invariant Extraction and Semantic Validation
Vadim Tynchenko, Igor Masich, Dmitry Martysyuk, Yury Kostromin, Aleksei Borodulin, Ivan Malashin, Vladimir Nelyub, and Andrei Gantimurov
(Bauman Moscow State Technical University, Russia)
Migrating legacy COBOL tax software to modern programming languages requires preserving business semantics in addition to producing syntactically correct translations. Small deviations in thresholds, deduction rules, arithmetic expressions, or boundary conditions may lead to incorrect financial computations. This paper presents TaxVerif, a verification-oriented migration framework that combines invariant extraction, assertion-guided translation, and mutation-based semantic validation. Candidate invariants are extracted from COBOL programs, translated into executable assertions, and used to verify Java-like translations under bounded execution. The evaluation includes an invariant-extraction benchmark, a controlled synthetic mutation benchmark, and a real-code benchmark using open-source COBOL programs with lexical, structural, and graph-based features. The results show that invariant-guided verification substantially improves semantic-error detection over syntax-based validation, while strict evaluation on unseen programs and unseen mutation types remains significantly more challenging, emphasizing the need for semantic verification in legacy-code migration.
Article Search
Article: asews26rasemain-id7-p (type: Full Paper (8 pages)) doi:10.1145/3820756.3844907
RASE '26: "Formal Analysis of Software ..."
Formal Analysis of Software Bills of Materials with Alloy
Sabato Nocera and Giuseppe Scanniello
(University of Salerno, Italy)
Software Bills of Materials (SBOMs) provide machine-readable inventories of the components constituting the software supply chains of software products. Their value depends on the quality of the information they contain, including component metadata and dependency relationships. Recent studies report quality concerns, such as incomplete information, which undermine downstream uses (e.g., vulnerability detection), but research on SBOM quality assurance remains limited. In this paper, we position the use of formal methods to analyze whether an SBOM satisfies intended quality requirements.
Article Search
Article: asews26rasemain-id26-p (type: Short Paper (4 pages)) doi:10.1145/3820756.3844915
RASE '26: "Minimizing Vulnerability Prevalence ..."
Minimizing Vulnerability Prevalence in Maven Dependencies via Answer Set Programming
Christian Macho, Laura Hofer, and Martin Pinzger
(University of Klagenfurt, Austria)
Software projects rely extensively on third-party libraries, but dependency versions with known vulnerabilities can expose them to security risks. Updating these dependencies is challenging because alternative versions may introduce different transitive dependencies, additional vulnerabilities, or compatibility failures. Moreover, not every dependency for which a vulnerability is reported is necessarily used by the project.
We present an approach that uses Answer Set Programming to identify less vulnerable dependency configurations for Apache Maven projects. The approach combines vulnerability information from three security advisories with the resolved dependency tree, alternative dependency subtrees, and the project's callgraph which identifies dependencies reached from the project. Clingo, our solver, then outputs ranked dependency configurations, reducing the number of vulnerabilities while maintaining buildability.
We evaluated our approach on twelve open-source Java projects and found that only ten of them actually use dependencies with vulnerabilities. For seven of the remaining ten projects, our approach identifies a buildable configuration with fewer vulnerabilities, reducing their number by 40.3% and their weighted vulnerability score by 41.4%. We also show that our approach is feasible in practice, yielding a median runtime of 15 and 2 minutes overall and for the optimization step only, respectively.
Article Search
Article: asews26rasemain-id27-p (type: Full Paper (8 pages)) doi:10.1145/3820756.3844916
RASE '26: "HermesAngler: Enabling Dynamic ..."
HermesAngler: Enabling Dynamic Instrumentation of iOS React Native Apps
Martin Renze and Fabian Scherf
(Fraunhofer SIT, Germany; ATHENE, Germany)
We present HermesAngler, a Frida-based dynamic instrumentation tool that enables function hooking inside the Hermes bytecode context of iOS React Native apps. We additionally present static analysis results concerning hardcoded secrets in a sample of 190 real-world React Native apps, which motivates the need for dynamic instrumentation at the Hermes bytecode level. We demonstrate HermesAngler on selected real-world apps, confirming statically identified weaknesses as proof-of-concept, such as the use of hardcoded AES keys at runtime.
Article Search
Artifacts Available
Article: asews26rasemain-id22-p (type: Short Paper (4 pages)) doi:10.1145/3820756.3844913
RASE '26: "Formal Analysis of Software ..."
Formal Analysis of Software Bills of Materials with Alloy
Sabato Nocera and Giuseppe Scanniello
(University of Salerno, Italy)
Software Bills of Materials (SBOMs) provide machine-readable inventories of the components constituting the software supply chains of software products. Their value depends on the quality of the information they contain, including component metadata and dependency relationships. Recent studies report quality concerns, such as incomplete information, which undermine downstream uses (e.g., vulnerability detection), but research on SBOM quality assurance remains limited. In this paper, we position the use of formal methods to analyze whether an SBOM satisfies intended quality requirements.
Article Search
Article: asews26rasemain-id26-p (type: Short Paper (4 pages)) doi:10.1145/3820756.3844915
RASE '26: "HermesAngler: Enabling Dynamic ..."
HermesAngler: Enabling Dynamic Instrumentation of iOS React Native Apps
Martin Renze and Fabian Scherf
(Fraunhofer SIT, Germany; ATHENE, Germany)
We present HermesAngler, a Frida-based dynamic instrumentation tool that enables function hooking inside the Hermes bytecode context of iOS React Native apps. We additionally present static analysis results concerning hardcoded secrets in a sample of 190 real-world React Native apps, which motivates the need for dynamic instrumentation at the Hermes bytecode level. We demonstrate HermesAngler on selected real-world apps, confirming statically identified weaknesses as proof-of-concept, such as the use of hardcoded AES keys at runtime.
Article Search
Artifacts Available
Article: asews26rasemain-id22-p (type: Short Paper (4 pages)) doi:10.1145/3820756.3844913
RASE '26: "Software Engineering Governance: ..."
Software Engineering Governance: Foundations for Trustworthy Automated Software Engineering
Carlos Mario Braga, Manuel A. Serrano, and Eduardo Fernandez-Medina
(University of Castilla-La Mancha, Spain)
Automated software engineering is progressively evolving from supporting human engineers to actively performing engineering activities through AI assistants, autonomous agents, automated testing, vulnerability repair, and deployment pipelines. As automation becomes an active participant in the software lifecycle, trustworthiness concerns extend beyond software systems themselves to the engineering processes, organisational decisions, and automated actors involved in their development. This evolution elevates Software Engineering Governance from a supporting organisational activity to a central engineering concern.
The established challenge of translating normative trustworthiness principles into operational practice becomes particularly consequential when automated systems perform engineering activities directly. Software Engineering Governance provides an engineering perspective for connecting these domains by interpreting trustworthiness principles within organisational and regulatory contexts, establishing operational lifecycle artefacts that guide automated execution, and providing the basis for interpreting the operational evidence generated during execution in subsequent accountable evaluation.
Viewing trustworthy automation through this governance perspective reveals two complementary governance responsibilities. Before automated execution, trustworthiness objectives must be systematically operationalised into lifecycle artefacts that guide or constrain engineering behaviour. After execution, operational evidence must be interpreted to support the accountable evaluation of observed engineering behaviour. Their combination provides the organising perspective through which Software Engineering Governance relates governance decisions made before execution to the evidence available for evaluation afterwards
Article Search
Article: asews26rasemain-id29-p (type: Full Paper (8 pages)) doi:10.1145/3820756.3844918
RASE '26: "How Grammar-Based Synthetic ..."
How Grammar-Based Synthetic Data Shapes Behavior Learning: An Empirical Study of Input Generation Strategies
Tural Mammadov, Alireza Kheradmand, Marius Smytzek, and Andreas Zeller
(CISPA Helmholtz Center for Information Security, Germany; Saarland University, Germany)
Grammar-based fuzzing can synthesize valid inputs for programs that process structured data. For behavior learning, the same inputs can be paired with the outputs a program produces, creating datasets that train neural program mocks. The fuzzer then serves as a data generator, and its selection policy determines which behaviors the model observes during training. To study how grammar-based input generation affects neural behavior modeling, we compare four generator configurations, train encoder-decoder Transformer models on 10,000 and 100,000 generated input-output pairs, and evaluate the resulting models on synthetic hold-out data and available real-world samples. On synthetic hold-out data, the larger training corpus improves the BLEU score in all 48 paired comparisons; random generation and k-path guidance are strongest on average, though the best generator remains subject-dependent. On real-world data, the larger corpus improves the BLEU score in 35 of 40 paired comparisons, but the gains are smaller and less consistent. JSON-to-YAML models retain high real-world accuracy, whereas msticpy and py-asciimath models expose large synthetic-to-real quality gaps. Overall, generator selection affects learned behavior models, but practical performance depends strongly on how well grammar-generated inputs match the deployment distribution.
Article Search
Article: asews26rasemain-id21-p (type: Full Paper (8 pages)) doi:10.1145/3820756.3844912
RASE '26: "When Log Parsing Removes What ..."
When Log Parsing Removes What Matters: Detecting Failures in Distributed GPU Workloads
Boyuan Cui, Yuyang Zeng, Yifang Tian, and Hans-Arno Jacobsen
(University of Toronto, Canada)
Large AI models increasingly rely on distributed GPU execution, yet it remains unclear whether established anomaly detection methods transfer effectively to this setting. We introduce GF-Bench, a PyTorch-based framework and dataset for generating controlled failures in distributed GPU workloads. The framework supports extensible workload and fault-injection logic; based on this design, the current release targets failures visible in multi-GPU communication logs and provides 220 labeled two-GPU executions covering normal runs, CUDA out-of-memory fail-stop failures, and persistent straggler fail-slow failures. Using GF-Bench, we evaluate three sequence-based methods (DeepLog, LogAnomaly, and LogBERT) and the LLM-based LogPrompt on parsed logs, and additionally evaluate LogPrompt on raw logs. On parsed inputs, no method reliably detects both failure classes, and the highest overall F1 score is 0.667. Raw inputs raise the overall F1 score of in-context Log- Prompt to 0.983, at 9.0× its average input-token usage and 8.7× its per-run cost. These results reveal a trade-off between effectiveness and efficiency: parsed inputs reduce LLM token use and inference cost but can discard the distributed execution context required for accurate failure detection. GF-Bench provides a reproducible basis for studying cost-efficient anomaly detection methods and can be extended to additional GPU workloads and failure types.
Article Search
Article: asews26rasemain-id32-p (type: Short Paper (4 pages)) doi:10.1145/3820756.3844919
RASE '26: "TaxVerif: Verification-Oriented ..."
TaxVerif: Verification-Oriented COBOL Tax-Code Migration via Invariant Extraction and Semantic Validation
Vadim Tynchenko, Igor Masich, Dmitry Martysyuk, Yury Kostromin, Aleksei Borodulin, Ivan Malashin, Vladimir Nelyub, and Andrei Gantimurov
(Bauman Moscow State Technical University, Russia)
Migrating legacy COBOL tax software to modern programming languages requires preserving business semantics in addition to producing syntactically correct translations. Small deviations in thresholds, deduction rules, arithmetic expressions, or boundary conditions may lead to incorrect financial computations. This paper presents TaxVerif, a verification-oriented migration framework that combines invariant extraction, assertion-guided translation, and mutation-based semantic validation. Candidate invariants are extracted from COBOL programs, translated into executable assertions, and used to verify Java-like translations under bounded execution. The evaluation includes an invariant-extraction benchmark, a controlled synthetic mutation benchmark, and a real-code benchmark using open-source COBOL programs with lexical, structural, and graph-based features. The results show that invariant-guided verification substantially improves semantic-error detection over syntax-based validation, while strict evaluation on unseen programs and unseen mutation types remains significantly more challenging, emphasizing the need for semantic verification in legacy-code migration.
Article Search
Article: asews26rasemain-id7-p (type: Full Paper (8 pages)) doi:10.1145/3820756.3844907
RASE '26: "Can We Trust Strong Teachers? ..."
Can We Trust Strong Teachers? Auditing Knowledge Distillation for Binary Code Similarity Detection
Minh Khang Van, Yunru Wang, and Johannes Kinder
(LMU Munich, Germany)
Large binary models are costly to use for Binary Code Similarity Detection (BCSD) at scale, motivating their distillation into compact encoders. Existing BCSD distillation work focuses mainly on teacher reliability, while student-side factors remain underexplored. We conduct a controlled study on a large-scale cross-project benchmark, fixing the student architecture and task objective while comparing three distillation objectives with the retrieval-specialized CLAP and the best-performing objective with the generative decompiler Nova. Our results show that direct task training produces a strong student that nearly matches zero-shot CLAP, while distillation adds only marginal gains. We therefore study two student-side factors that shape how teacher supervision is received: its interaction with task training and the input visible to the student. Under the same loss coefficient, different distillation objectives produce markedly different gradient interactions with InfoNCE, yet these statistics do not yield a simple rule for loss balancing. Increasing the student context from 64 to 1,024 tokens consistently improves embedding distillation and narrows the gap to CLAP. These results show that reliable BCSD distillation depends not only on teacher quality, but also on the student's optimization and input constraints.
Article Search
Article: asews26rasemain-id28-p (type: Full Paper (Replication Study 8 pages)) doi:10.1145/3820756.3844917
RASE '26: "Can We Trust Strong Teachers? ..."
Can We Trust Strong Teachers? Auditing Knowledge Distillation for Binary Code Similarity Detection
Minh Khang Van, Yunru Wang, and Johannes Kinder
(LMU Munich, Germany)
Large binary models are costly to use for Binary Code Similarity Detection (BCSD) at scale, motivating their distillation into compact encoders. Existing BCSD distillation work focuses mainly on teacher reliability, while student-side factors remain underexplored. We conduct a controlled study on a large-scale cross-project benchmark, fixing the student architecture and task objective while comparing three distillation objectives with the retrieval-specialized CLAP and the best-performing objective with the generative decompiler Nova. Our results show that direct task training produces a strong student that nearly matches zero-shot CLAP, while distillation adds only marginal gains. We therefore study two student-side factors that shape how teacher supervision is received: its interaction with task training and the input visible to the student. Under the same loss coefficient, different distillation objectives produce markedly different gradient interactions with InfoNCE, yet these statistics do not yield a simple rule for loss balancing. Increasing the student context from 64 to 1,024 tokens consistently improves embedding distillation and narrows the gap to CLAP. These results show that reliable BCSD distillation depends not only on teacher quality, but also on the student's optimization and input constraints.
Article Search
Article: asews26rasemain-id28-p (type: Full Paper (Replication Study 8 pages)) doi:10.1145/3820756.3844917
RASE '26: "How Grammar-Based Synthetic ..."
How Grammar-Based Synthetic Data Shapes Behavior Learning: An Empirical Study of Input Generation Strategies
Tural Mammadov, Alireza Kheradmand, Marius Smytzek, and Andreas Zeller
(CISPA Helmholtz Center for Information Security, Germany; Saarland University, Germany)
Grammar-based fuzzing can synthesize valid inputs for programs that process structured data. For behavior learning, the same inputs can be paired with the outputs a program produces, creating datasets that train neural program mocks. The fuzzer then serves as a data generator, and its selection policy determines which behaviors the model observes during training. To study how grammar-based input generation affects neural behavior modeling, we compare four generator configurations, train encoder-decoder Transformer models on 10,000 and 100,000 generated input-output pairs, and evaluate the resulting models on synthetic hold-out data and available real-world samples. On synthetic hold-out data, the larger training corpus improves the BLEU score in all 48 paired comparisons; random generation and k-path guidance are strongest on average, though the best generator remains subject-dependent. On real-world data, the larger corpus improves the BLEU score in 35 of 40 paired comparisons, but the gains are smaller and less consistent. JSON-to-YAML models retain high real-world accuracy, whereas msticpy and py-asciimath models expose large synthetic-to-real quality gaps. Overall, generator selection affects learned behavior models, but practical performance depends strongly on how well grammar-generated inputs match the deployment distribution.
Article Search
Article: asews26rasemain-id21-p (type: Full Paper (8 pages)) doi:10.1145/3820756.3844912
RASE '26: "When Log Parsing Removes What ..."
When Log Parsing Removes What Matters: Detecting Failures in Distributed GPU Workloads
Boyuan Cui, Yuyang Zeng, Yifang Tian, and Hans-Arno Jacobsen
(University of Toronto, Canada)
Large AI models increasingly rely on distributed GPU execution, yet it remains unclear whether established anomaly detection methods transfer effectively to this setting. We introduce GF-Bench, a PyTorch-based framework and dataset for generating controlled failures in distributed GPU workloads. The framework supports extensible workload and fault-injection logic; based on this design, the current release targets failures visible in multi-GPU communication logs and provides 220 labeled two-GPU executions covering normal runs, CUDA out-of-memory fail-stop failures, and persistent straggler fail-slow failures. Using GF-Bench, we evaluate three sequence-based methods (DeepLog, LogAnomaly, and LogBERT) and the LLM-based LogPrompt on parsed logs, and additionally evaluate LogPrompt on raw logs. On parsed inputs, no method reliably detects both failure classes, and the highest overall F1 score is 0.667. Raw inputs raise the overall F1 score of in-context Log- Prompt to 0.983, at 9.0× its average input-token usage and 8.7× its per-run cost. These results reveal a trade-off between effectiveness and efficiency: parsed inputs reduce LLM token use and inference cost but can discard the distributed execution context required for accurate failure detection. GF-Bench provides a reproducible basis for studying cost-efficient anomaly detection methods and can be extended to additional GPU workloads and failure types.
Article Search
Article: asews26rasemain-id32-p (type: Short Paper (4 pages)) doi:10.1145/3820756.3844919