Powered by
Conference Publishing Consulting

9th International Workshop on Automation of Software Test (AST 2014), May 31 – June 1, 2014, Hyderabad, India

AST 2014 – Proceedings

Contents - Abstracts - Authors

9th International Workshop on Automation of Software Test (AST 2014)

Frontmatter

Title Page


Message from the Chairs
Welcome to AST 2014—The 9th International Workshop on Automation of Software Test, held in conjunction with the 36th International Conference on Software Engineering (ICSE 2014) in Hyderabad, India, May 31 – June 1, 2014.
The workshop is the 9th edition of the successful AST workshops held at ICSE 2006–2013. It continues the tradition of the previous workshops by providing a forum for both researchers and practitioners to exchange recent research results and novel ideas, identify challenging and new emerging problems in research and practice, and formulate visions for the future in software test automation. The workshop aims at bridging the gap between theory and practice in order to improve the current state of the practice and foster innovative research in the area.

Security and Performance Testing

Attack Pattern-Based Combinatorial Testing
Josip Bozic, Dimitris E. Simos, and Franz Wotawa
(Graz University of Technology, Austria; SBA Research, Austria)
The number of potential security threats rises with the increasing number of web applications, which cause tremendous financial and existential implications for developers and users as well. The biggest challenge for security testing is to specify and implement ways in order to detect potential vulnerabilities of the developed system in a never ending quest against new security threats but also to cover already known ones so that a program is suited against typical attack vectors. For these purposes many approaches have been developed in the area of model-based security testing in order to come up with solutions for real-world application problems. These approaches provide theoretical background as well as practical solutions for certain security issues. In this paper, we partially rely on previous work but focus on the representation of attack patterns using UML state diagrams. We extend previous work in combining the attack pattern models with combinatorial testing in order to provide concrete test input, which is submitted to the system under test. With combinatorial testing we capture different combinations of inputs and thus increasing the likelihood to find weaknesses in the implementation under test that can be exploited. Besides the foundations of our approach we further report on first experiments that indicate its practical use.

An Automated Testing Approach for Inter-application Security in Android
Chenkai Guo, Jing Xu, Hongji Yang, Ying Zeng, and Shuang Xing
(Nankai University, China; Bath Spa University, UK)
Recently, Google Android has occupied a major market share of mobile phone systems as a result of its openness for developers and richness for users. By the distribution channels of the Android market, both development and use of Android applications soar. However, the low development threshold of applications leads to weak security awareness of developers. Moreover, Android applications lack strict security standards, resulting that security crisis has become increasingly prominent. For now, an application's biggest security threat falls on its messaging mechanism between components. Once permission’s verification is neglected, it is easy to be exploited by attackers, causing immeasurable loss. We analyze the security mechanism of Android inter-application components, and accordingly construct the security rules. Specifically, a compositional approach including static and dynamic automated testing techniques is proposed to detect the security vulnerabilities caused by messaging between components. In our approach, the static part obtains rough results and some parameter information. After that, the dynamic part automatically generates attack cases for verifying these results. This approach can be used not only to discover potential weaknesses within inter-application components but also to automatically simulate attack behaviors. Thereby, the detection results’ effectiveness can be verified.

Categorizing Configuration Parameters of Smartphones for Energy Performance Testing
Kshirasagar Naik, Yasir Ali, Veluppillai Mahinthan, Ajit Singh, and Abdulhakim Abogharaf
(University of Waterloo, Canada; Aljabal Algharby University, Libya)
Energy performance testing in smartphones is a challenging task and the extent of exhaustive testing depends on the system configurations for different parameters and applications. In this paper, we propose a technique to classify the configuration parameters of a smartphone by partitioning them into two groups based on their maximum differential power (impact on power consumption). We validate the technique by applying it to four different smartphones: BlackBerry Bold 9700, BlackBerry Z10, Apple iPhone 3GS and Samsung Galaxy Nexus. The four devices represent a wide spectrum of devices with four operating systems (BB7, BB10, iOS, and Android), three makers (BlackBerry, Apple and Samsung), four hardware platforms, and relatively old and new devices.

Test and Change Coverage

Social Coverage for Customized Test Adequacy and Selection Criteria
Breno Miranda and Antonia Bertolino
(University of Pisa, Italy; ISTI-CNR, Italy)
Test coverage information can be very useful for guiding testers in enhancing their test suites to exercise possible uncovered entities and in deciding when to stop testing. However, for complex applications that are reused in different contexts and for emerging paradigms (e.g., component-based development, service-oriented architecture, and cloud computing), traditional coverage metrics may no longer provide meaningful information to help testers on these tasks. Various proposals are advocating to leverage information that come from the testing community in a collaborative testing approach. In this work we introduce a coverage metric, the Social Coverage, that customizes coverage information in a given context based on coverage data collected from similar users. To evaluate the potential of our proposed approach, we instantiated the social coverage metric in the context of a real world service oriented application. In this exploratory study, we were able to predict the entities that would be of interest for a given user with an average precision of 97% and average recall of 75%. Our results suggest that, in similar environments, social coverage can provide a better support to testers than traditional coverage.

Selecting Manual Regression Test Cases Automatically using Trace Link Recovery and Change Coverage
Sebastian Eder, Benedikt Hauptmann, Maximilian Junker, Rudolf Vaas, and Karl-Heinz Prommer
(TU München, Germany; Munich Re, Germany)
Regression tests ensure that existing functionality is not impaired by changes to an existing software system. However, executing complete test suites often takes much time. Therefore, a subset of tests has to be found that is sufficient to validate whether the system under test is still valid after it has been changed. This test case selection is especially important if regression tests are executed manually, since manual execution is time intensive and costly. To select manual test cases, many regression testing techniques exist that aim on achieving coverage of changed or even new code. Many of these techniques base on coverage data from prior test runs or logical properties such as annotated pre and post conditions in the source code. However, coverage information becomes outdated if a system is changed extensively. Also annotated logical properties are often not available in industrial software systems. We present an approach for test selection that is based on static analyses of the test suite and the system's source code. We combine trace link recovery using latent semantic indexing with the metric change coverage, which assesses the coverage of source code changes. The proposed approach works automatically without the need to execute tests beforehand or annotate source code. Furthermore, we present a first evaluation of the approach.

Effective Unit-Testing in Model-Based Software Development
Damodaram Kamma and Pooja Maruthi
(Bosch, India)
Model-based software development is extensively used in avionics and automotive safety critical control software applications. In model-based software development, highly optimized code is generated automatically from models. Such code is often hard to understand and this can make it difficult to write test cases. Therefore, in model based software development, test cases have to be derived based on the models to achieve coverage of code auto-generated from the models. Further, safety standards in those domains often demand effective unit-testing method to check functional requirements as well as achieve 100% code coverage.
In this paper, we first discuss three methods for unit testing in model based software development, namely Modified Condition & Decision Coverage (MCDC), Classification tree and Exploratory methods. We then discuss results of our field study conducted on 3 live projects at Robert Bosch Engineering & Business Solutions Limited to check on the effectiveness of three approaches. Based on the results from our field study, we conclude that MCDC method along with boundary value analysis is most productive to check functional requirements as well as achieve 100% coverage of auto-generated code.

Property Checking and Debugging

Verification of Non-functional Properties of Cloud-Based Distributed System Services
Kaliappa Ravindran and Arun Adiththan
(City University of New York, USA)
For distributed system services implemented on a cloud, system verification assumes added importance because of third-party control of cloud resources and the attendant problems of faults, QoS degradations, and security violations. Our paper focuses on a "model-based assessment" to reason about the non-functional properties of a cloud-based distributed system using observational agents. Our approach is corroborated by measurements on system-level prototypes and simulation analysis of system models in the face of hostile environment conditions. A case study of CDN realized on cloud infrastructures is also described.

Improved Semantics and Implementation through Property-Based Testing with QuickCheck
Huiqing Li and Simon Thompson
(University of Kent, UK)
Testing is the primary method to validate that a software implementation meets its specification. In this paper, we demonstrate an approach to validating an executable semantics using property- and model-based random testing in QuickCheck to automate and unify the testing of the semantics and its implementation. Our approach shows the use of executable semantics to bridge the gap between formal mathematical specification and implementation, as well as emphasising the suitability of functional programming languages -- in this case Erlang -- for writing executable semantics.
The approach is illustrated through a concrete example, in which the implementation of a proposed extension to the Erlang programming language -- scalable groups -- is tested. This new component comes with a small-step operational semantics written in mathematical notation, and was initially tested using unit testing. Through our work, we were able to find new bugs in both the implementation and the specification.

Reduce First, Debug Later
Alexander Elyasov, Wishnu Prasetya, Jurriaan Hage, and Andreas Nikas
(Utrecht University, Netherlands)
The delta debugging minimization algorithm ddmin provides an efficient procedure for the simplification of failing test-cases. Despite its contribution towards the automation of debugging, ddmin still requires a significant number of iterations to complete. The delta debugging (DD) search space can be narrowed down by providing the test-case circumstances that are most likely relevant to the occurred failure. This paper proposes a novel approach to the problem of failure simplification consisting of two consecutive phases: 1) failure reduction by rewriting (performed offline), and 2) DD invocation (performed online). In the best case scenario, the reduction phase may already deliver a simplified failure, otherwise, it potentially supplies DD with extra information about where to look for the failure. The proposed solution has been prototyped as a web application debugging tool, which was evaluated on a shopping cart web application - Flex Store. The evaluation shows an improvement of the DD execution time if the offline reduction over-approximates the failure.

proc time: 0.72