SANER 2020 Workshops
Workshops of the 2020 IEEE 27th International Conference on Software Analysis, Evolution, and Reengineering (SANER)
Powered by
Conference Publishing Consulting

2020 IEEE 3rd International Workshop on Validation, Analysis, and Evolution of Software Tests (VST), February 18, 2020, London, ON, Canada

VST 2020 – Proceedings

Contents - Abstracts - Authors

2020 IEEE 3rd International Workshop on Validation, Analysis, and Evolution of Software Tests (VST)


Title Page

Welcome from the Chairs
Welcome to the 3rd Workshop on Validation, Analysis and Evolution of Software Tests (VST 2020). The workshop takes place in London, Ontario, Canada, on February 18, 2020, co-located with SANER 2020, the 27th IEEE International Conference on Software Analysis, Evolution and Reengineering.


Semi-automatic Test Case Expansion for Mutation Testing
Zhong Xi Lu, Sten Vercammen, and Serge Demeyer
(University of Antwerp, Belgium)
Mutation testing is the state-of-the-art technique for detecting weaknesses in a test suite. Unfortunately, alleviating these weakness (i.e. “killing the surviving mutants”) is quite labour-intensive. In this paper we investigate a recommender system which expands test cases with extra asserts for the easy-to-fix mutants. We evaluated a proof-of-concept tool on ten open-source projects, and killed up to 6% of the surviving mutants. This illustrates that such a test expansion system would free up valuable time to address the harder-to-fix mutants.

Article Search
An Empirical Evaluation for Object Initialization of Member Variables in Unit Testing
Stefan Fischer, Evelyn Nicole Haslinger, Markus Zimmermann, and Hannes Thaller
(Software Competence Center Hagenberg, Austria; Symflower, Austria; JKU Linz, Austria)
Automated test case generation techniques usually aim to maximize some coverage criteria. For object oriented languages, like Java, the branches that can be reached in source code, frequently depend on the internal object state. Meaning certain branches will only be taken, if fields inside the tested class are set to specific values. It is however not obvious how much of the internal object state can be controlled. In this paper, we analyzed a corpus of 110 open source systems to evaluate how settable their classes are. Meaning we looked for ways that fields inside classes can be written. For instance, we analyzed the source code to identify setter methods that can be used to set the values of a field. Our results show that 66.5% of fields can be set to a desired value, while 31.5% of fields may be only settable to particular values or require a more in depth analysis. Only for 2% of fields, we did not find any way to set their values.

Article Search
An Early Investigation of Unit Testing Practices of Component-Based Software Systems
Georg Buchgeher, Stefan Fischer, Michael Moser, and Josef Pichler
(Software Competence Center Hagenberg, Austria; University of Applied Sciences Upper Austria, Austria)
Component-based software development (CBSD) is one of the main programming paradigms of the last decades. The main idea of CBSD is to modularize a system as a configuration of multiple interacting components. Components interact with each other via dedicated component interfaces hiding a component's implementation and making components interchangeable. In this paper, we present an early investigation of unit testing practices of open source component-based software systems with the goal to find out how component-based software systems are actually tested and how to improve unit testing practices as part of future research. Our preliminary results show that unit tests typically directly test the component implementation and not dedicated component APIs. The method coverage of component APIs varied between 17% and 34% in the analyzed projects.

Article Search
Do Bug-Fix Types Affect Spectrum-Based Fault Localization Algorithms’ Efficiency?
Attila Szatmári, Béla Vancsics, and Árpád Beszédes
(University of Szeged, Hungary)
Finding a bug in the software is an expensive task, however, debugging is a crucial part of the software development life cycle. Spectrum-Based Fault Localization (SBFL) algorithms can reduce the time spent with debugging. Despite the fact that SBFL is a very well researched topic, there are not many tools that implement it. Many studies have dealt with the effectiveness of SBFL algorithms, although these have been evaluated on Java and C++ programming languages. We performed an empirical study on JavaScript programs (using BugsJS benchmark) to evaluate the relationship between algorithms efficiency and the bug-fix types. First we implemented three popular SBFL approaches, i.e. Tarantula, Ochiai and DStar, then examined whether there was a correlation/connection between the positions of the faulty methods in the suspiciousness ranks and bug-fix types. Results show that certain bug-fix types can be significantly differentiated from the others (in both positive and negative direction) based on the fault localization effectiveness of the investigated algorithms.

Article Search
Towards Fault Localization via Probabilistic Software Modeling
Hannes Thaller, Lukas Linsbauer, Alexander Egyed, and Stefan Fischer
(JKU Linz, Austria; Software Competence Center Hagenberg, Austria)
Software testing helps developers to identify bugs. However, awareness of bugs is only the first step. Finding and correcting the faulty program components is equally hard and essential for high-quality software. Fault localization automatically pinpoints the location of an existing bug in a program. It is a hard problem, and existing methods are not yet precise enough for widespread industrial adoption. We propose fault localization via Probabilistic Software Modeling (PSM). PSM analyzes the structure and behavior of a program and synthesizes a network of Probabilistic Models (PMs). Each PM models a method with its inputs and outputs and is capable of evaluating the likelihood of runtime data. We use this likelihood evaluation to find fault locations and their impact on dependent code elements. Results indicate that PSM is a robust framework for accurate fault localization.

Article Search
Simulating the Effect of Test Flakiness on Fault Localization Effectiveness
Béla Vancsics, Tamás Gergely, and Árpád Beszédes
(University of Szeged, Hungary)
Test flakiness (non-deterministic behavior of test cases) is an increasingly serious concern in industrial practice. However, there are relatively little research results available that systematically address the analysis and mitigation of this phenomena. The dominant approach to handle flaky tests is still detecting and removing them from automated test executions. However, some reports showed that the amount of flaky test is in many cases so high that we should rather start working on approaches that operate in the presence of flaky tests. In this work, we investigate how test flakiness affects the effectiveness of Spectrum Based Fault Localization (SBFL), a popular class of software Fault Localization (FL), which heavily relies on test case execution outcomes. We performed a simulation based experiment to find out what is the relationship between the level of test flakiness and fault localization effectiveness. Our results could help the users of automated FL methods to understand the implications of flaky tests in this area and to design novel FL algorithms that take into account test flakiness.

Article Search

proc time: 1.02