Powered by
1st International Workshop on Automated Techniques for Integrity and Quality in Software-Engineering Research (ATIQSER 2026), October 12–16, 2026,
Munich, Germany
1st International Workshop on Automated Techniques for Integrity and Quality in Software-Engineering Research (ATIQSER 2026)
Frontmatter
Title Page
Article: asews26atiqserforeword-fm000-p (type: Frontmatter) doi:
Welcome from the Chairs
As powerful tools increasingly streamline how software engineering (SE) research is conducted and reported, submission volumes at top-tier SE venues are surging. To ensure this growth does not compromise research quality, scalable and reliable automated techniques are essential to support peer review and publishing practices. These range from traditional methods—such as validating bibliography integrity against public databases (e.g., the hallucinator tool)—to AI-assisted review support (as used at AAAI) and administrative automation (e.g., TPMS for paper bidding and assignment).
ATIQSER provides a dedicated platform for researchers developing, evaluating, or utilizing these techniques.
Article: asews26atiqserforeword-fm001-p (type: Frontmatter) doi:
Papers
doi-checker: A Tool for Validating References of Research Papers
Dirk Beyer and
Marian Lingsch-Rosenfeld
(LMU Munich, Germany)
The reference list of a research paper links its claims to prior work.
However, they frequently contain references without digital object
identifiers (DOIs), invalid DOIs, or valid DOIs pointing to a different publication. The last case is a threat to academic integrity. To
improve upon this situation, we present doi-checker, a tool that
extracts the references of a paper, resolves their metadata against
the Crossref and DataCite registries, and reports for each reference whether its DOI is missing, invalid, or inconsistent with the
cited work. In addition, doi-checker also checks the metadata of
each reference for consistency with the listed authors. We evaluate
doi-checker on all the available open-access proceedings of six major conferences (CAV, TACAS, POPL, PLDI, FSE, and ISSTA) from
2020 to 2026, covering three page formats common in computer science: LNCS, ACM double column, and ACM single column. Across
the 193 770 references of these 3 888 papers, the share for which
doi-checker finds DOIs rises from 26.0 % in 2020 to 60.7 % in 2026.
Of these, 0.47 % do not resolve, and for 5.79 %, the registered metadata disagrees with the cited entry. Setting aside a single volume
for which reference extraction is poor, we find no evidence that
such disagreements concentrate in recent years.
@InProceedings{ATIQSER26p1,
author = {Dirk Beyer and Marian Lingsch-Rosenfeld},
title = {doi-checker: A Tool for Validating References of Research Papers},
booktitle = {Proc.\ ATIQSER},
publisher = {ACM},
pages = {1-7},
doi = {10.1145/3844135.3845864},
year = {2026},
}
Publisher's Version
Published Artifact
Artifacts Available
Article: asews26atiqsermain-p1-p (type: Full Paper) doi:10.1145/3844135.3845864
ATIBA: Grounded Integrity and Quality Checking for Research Papers
Veli Karakaya,
Semih Çağlar,
Yusuf Yiğit Korkmaz, and
Eray Tüzün
(Bilkent University, Türkiye)
Checking a manuscript's reference integrity, its compliance with a target venue's specific submission rules, and its adherence to community reporting standards is manual, repetitive, and different for every venue so in practice it is done inconsistently or skipped. We present ATIBA, a tool that runs five grounded integrity and quality checks on a manuscript: a reference-integrity check that verifies each citation against bibliographic sources and flags retracted or unfindable references; a venue/track compliance check that derives submission criteria directly from a venue's own call-for-papers page and evaluates the manuscript against them, each verdict anchored to a verbatim quote from that page; an empirical-standards compliance check against the ACM SIGSOFT Empirical Standards, with a hallucination defence that discards any evidence quote it cannot locate verbatim in the manuscript; a multi-mode AI review (venue-specific, formal, and page-anchored annotation) powered by GPT-5.4 through Azure OpenAI; and a citation-suggestion feature that proposes candidate references for a manuscript and verifies each against bibliographic sources before it is shown to the user. All five checks are designed around the same principle: an LLM is only trusted to judge, never to invent the evidence it judges against. We evaluated ATIBA through a moderated user study with 13 non-author participants. Agreement across the six survey items ranged from 69% to 92%, with a mean of 85%, providing initial evidence of positive perceived usefulness across the evaluated workflows. These findings establish perceived usefulness; objective accuracy remains to be measured.
@InProceedings{ATIQSER26p8,
author = {Veli Karakaya and Semih Çağlar and Yusuf Yiğit Korkmaz and Eray Tüzün},
title = {ATIBA: Grounded Integrity and Quality Checking for Research Papers},
booktitle = {Proc.\ ATIQSER},
publisher = {ACM},
pages = {8-14},
doi = {10.1145/3844135.3845865},
year = {2026},
}
Publisher's Version
Article: asews26atiqsermain-p2-p (type: Full Paper) doi:10.1145/3844135.3845865
RefVerifier: Semi-automated Reference Claim Verification for Scientific Manuscripts
Stefania Mocan,
Florian Angermeir, and
Mark Kreitz
(TU Munich, Germany; fortiss, Germany; Blekinge Institute of Technology, Sweden; University of the Bundeswehr Munich, Germany)
As software engineering research submission counts surge, peer reviewers face severe time constraints, making systematic verification of citation-supported claims prohibitively expensive. Consequently, unsubstantiated claims and semantic drift can propagate undetected across scientific literature. Existing approaches such as fact-checking and retrieval-augmented generation tools operate on open-domain web data or evaluate claims in isolation without processing complete manuscripts. To address this gap, we present RefVerifier, a semi-automated, citation-bounded reference verification prototype designed to support in academic peer review. RefVerifier extracts citation-bearing sentences from manuscripts, checks bibliography metadata against scholarly databases, resolves references to full-text open-access PDFs, localizes relevant evidence passages, and generates verdicts with natural language explanations. Evaluating RefVerifier on public benchmarks shows claim detection at an F1 score of 0.990, open-access resolution of 57.6% of references, and evidence localization with a hit rate of 98% on abstracts and 68% on complete cited papers. In an end-to-end test with eight manuscripts, RefVerifier achieves a verdict accuracy of 71%. By automating document retrieval and evidence localization while preserving reviewer oversight, RefVerifier provides first indicators for the feasibility of semi-automated integrity checks in scholarly publishing.
@InProceedings{ATIQSER26p15,
author = {Stefania Mocan and Florian Angermeir and Mark Kreitz},
title = {RefVerifier: Semi-automated Reference Claim Verification for Scientific Manuscripts},
booktitle = {Proc.\ ATIQSER},
publisher = {ACM},
pages = {15-21},
doi = {10.1145/3844135.3845866},
year = {2026},
}
Publisher's Version
Article: asews26atiqsermain-p3-p (type: Full Paper) doi:10.1145/3844135.3845866
proc time: 0.02