Powered by
2014 International Symposium on Software Testing and Analysis (ISSTA),
July 21–25, 2014,
San Jose, CA, USA
Tool Demonstrations
Wed, Jul 23, 18:00 - 22:00, Winchester Room
Legend: An Agile DSL Toolset for Web Acceptance Testing
Tariq M. King, Gabriel Nunez, Dionny Santiago, Adam Cando, and Cody Mack
(Ultimate Software, USA)
Agile development emphasizes collaborations among customers, business analysts, domain experts, developers, and testers. However, the large scale and rapid pace of many agile projects presents challenges during testing activities. Large sets of test artifacts must be comprehensible and available to various stakeholders, traceable to requirements, and easily maintainable as the software evolves. In this paper we describe Legend, a toolset that leverages domain-specific language to streamline functional testing in agile projects. Some key features of the toolset include test template generation from user stories, model-based automation, test inventory synchronization, and centralized test tagging.
@InProceedings{ISSTA14p409,
author = {Tariq M. King and Gabriel Nunez and Dionny Santiago and Adam Cando and Cody Mack},
title = {Legend: An Agile DSL Toolset for Web Acceptance Testing},
booktitle = {Proc.\ ISSTA},
publisher = {ACM},
pages = {409--412},
doi = {},
year = {2014},
}
Video
ProCrawl: Mining Test Models from Multi-user Web Applications
Matthias Schur, Andreas Roth, and
Andreas Zeller
(SAP, Germany; Saarland University, Germany)
Today's web applications demand very high release cycles--and consequently, frequent tests. Automating these tests typically requires a behavior model: A description of the states the application can be in, the transitions between these states, and the expected results. Furthermore one needs scripts to make the abstract actions (transitions) in the model executable. However, specifying such behavior models and writing the necessary scripts manually is a hard task. We present ProCrawl (Process Crawler), a tool that automatically mines (extended) finite-state machines from (multi-user) web applications and generates executable test scripts. ProCrawl explores the behavior of the application by systematically generating program runs and observing changes on the application's user interface. The resulting models can be directly used for effective model-based testing, in particular regression testing.
@InProceedings{ISSTA14p413,
author = {Matthias Schur and Andreas Roth and Andreas Zeller},
title = {ProCrawl: Mining Test Models from Multi-user Web Applications},
booktitle = {Proc.\ ISSTA},
publisher = {ACM},
pages = {413--416},
doi = {},
year = {2014},
}
X-PERT: A Web Application Testing Tool for Cross-Browser Inconsistency Detection
Shauvik Roy Choudhary, Mukul R. Prasad, and
Alessandro Orso
(Georgia Tech, USA; Fujitsu Labs, USA)
Web applications are popular among developers because of the ease of development and deployment through the ubiquitous web browsing platform. However, differences in a web application's execution across different web browsers manifest as Cross-browser Inconsistencies (XBIs), which are a serious concern for web developers. Testing for XBIs manually is a laborious and error-prone process. In this demo we present X-PERT, which is a tool to identify XBIs in web applications automatically, without requiring any effort from the developer. X-PERT implements a comprehensive technique to identify XBIs and has been found to be effective in detecting real-world XBIs in our empirical evaluation. The source code of X-PERT and XBI reports from our evaluation are available at http://gatech.github.io/xpert.
@InProceedings{ISSTA14p417,
author = {Shauvik Roy Choudhary and Mukul R. Prasad and Alessandro Orso},
title = {X-PERT: A Web Application Testing Tool for Cross-Browser Inconsistency Detection},
booktitle = {Proc.\ ISSTA},
publisher = {ACM},
pages = {417--420},
doi = {},
year = {2014},
}
Video
Info
Extending a Search-Based Test Generator with Adaptive Dynamic Symbolic Execution
Juan Pablo Galeotti, Gordon Fraser, and
Andrea Arcuri
(Saarland University, Germany; University of Sheffield, UK; Simula Research Laboratory, Norway)
Automatic unit test generation aims to support developers by alleviating the burden of test writing. Different techniques have been proposed over the years, each with distinct limitations. To overcome these limitations, we present an extension to the EvoSuite unit test generator that combines two of the most popular techniques for test case generation: Search-Based Software Testing (SBST) and Dynamic Symbolic Execution (DSE). A novel integration of DSE as a step of local improvement in a genetic algorithm results in an adaptive approach, such that the best test generation technique for the problem at hand is favoured, resulting in overall higher code coverage.
@InProceedings{ISSTA14p421,
author = {Juan Pablo Galeotti and Gordon Fraser and Andrea Arcuri},
title = {Extending a Search-Based Test Generator with Adaptive Dynamic Symbolic Execution},
booktitle = {Proc.\ ISSTA},
publisher = {ACM},
pages = {421--424},
doi = {},
year = {2014},
}
Canalyze: A Static Bug-Finding Tool for C Programs
Zhenbo Xu,
Jian Zhang , Zhongxing Xu, and Jiteng Wang
(University of Science and Technology of China, China; Institute of Software at Chinese Academy of Sciences, China; Beijing University of Posts and Telecommunications, China)
Symbolic analysis is a commonly used approach for static bug finding. It usually performs a precise path-by-path symbolic simulation from program inputs. A major challenge is its scalability and precision on interprocedural analysis. The former limits the application to large programs. The latter may lead to many false alarms.
This paper presents a flexible, scalable and practical static bug detection tool, called Canalyze, for C programs. The flexibility is embodied in our modular design that supports
different precision-level constraint solvers and interprocedural analyses. Based on these options, one can enable the less precise options to achieve a more scalable analysis or the more time-consuming options to perform a more precise analysis. Our tool is also practical to analyze real-world applications. It has been applied to some industry systems and open source programs like httpd, lighttpd, etc. And hundreds of newly found bugs were confirmed by the maintainers of our benchmarks.
@InProceedings{ISSTA14p425,
author = {Zhenbo Xu and Jian Zhang and Zhongxing Xu and Jiteng Wang},
title = {Canalyze: A Static Bug-Finding Tool for C Programs},
booktitle = {Proc.\ ISSTA},
publisher = {ACM},
pages = {425--428},
doi = {},
year = {2014},
}
Video
Info
MuCheck: An Extensible Tool for Mutation Testing of Haskell Programs
Duc Le,
Mohammad Amin Alipour,
Rahul Gopinath, and
Alex Groce
(Oregon State University, USA)
This paper presents MuCheck, a mutation testing tool for Haskell
programs. MuCheck is a counterpart to the widely used QuickCheck
random testing tool for functional programs, and can be used to
evaluate the efficacy of QuickCheck property definitions. The tool
implements mutation operators that are specifically designed for
functional programs, and makes use of the type system of Haskell to
achieve a more relevant set of mutants than otherwise
possible. Mutation coverage is particularly valuable for functional
programs due to highly compact code, referential transparency, and
clean semantics; these make augmenting a test suite or specification
based on surviving mutants a practical method for improved testing.
@InProceedings{ISSTA14p429,
author = {Duc Le and Mohammad Amin Alipour and Rahul Gopinath and Alex Groce},
title = {MuCheck: An Extensible Tool for Mutation Testing of Haskell Programs},
booktitle = {Proc.\ ISSTA},
publisher = {ACM},
pages = {429--432},
doi = {},
year = {2014},
}
The Major Mutation Framework: Efficient and Scalable Mutation Analysis for Java
René Just
(University of Washington, USA)
Mutation analysis seeds artificial faults (mutants) into a pro-
gram and evaluates testing techniques by measuring how
well they detect those mutants. Mutation analysis is well-
established in software engineering research but hardly used
in practice due to inherent scalability problems and the lack
of proper tool support. In response to those challenges, this
paper presents Major, a framework for mutation analysis
and fault seeding. Major provides a compiler-integrated mu-
tator and a mutation analyzer for JUnit tests.
Major implements a large set of optimizations to enable
efficient and scalable mutation analysis of large software sys-
tems. It has already been applied to programs with more
than 200,000 lines of code and 150,000 mutants. Moreover,
Major features its own domain specific language and is de-
signed to be highly configurable to support fundamental re-
search in software engineering. Due to its efficiency and
flexibility, the Major mutation framework is suitable for the
application of mutation analysis in research and practice. It
is publicly available at http://mutation-testing.org.
@InProceedings{ISSTA14p433,
author = {René Just},
title = {The Major Mutation Framework: Efficient and Scalable Mutation Analysis for Java},
booktitle = {Proc.\ ISSTA},
publisher = {ACM},
pages = {433--436},
doi = {},
year = {2014},
}
Defects4J: A Database of Existing Faults to Enable Controlled Testing Studies for Java Programs
René Just
, Darioush Jalali, and
Michael D. Ernst
(University of Washington, USA)
Empirical studies in software testing research may not be
comparable, reproducible, or characteristic of practice. One
reason is that real bugs are too infrequently used in software
testing research. Extracting and reproducing real bugs is
challenging and as a result hand-seeded faults or mutants
are commonly used as a substitute.
This paper presents Defects4J, a database and extensible
framework providing real bugs to enable reproducible studies
in software testing research. The initial version of Defects4J
contains 357 real bugs from 5 real-world open source pro-
grams. Each real bug is accompanied by a comprehensive
test suite that can expose (demonstrate) that bug. Defects4J
is extensible and builds on top of each program’s version con-
trol system. Once a program is configured in Defects4J, new
bugs can be added to the database with little or no effort.
Defects4J features a framework to easily access faulty and
fixed program versions and corresponding test suites. This
framework also provides a high-level interface to common
tasks in software testing research, making it easy to con-
duct and reproduce empirical studies. Defects4J is publicly
available at http://defects4j.org.
@InProceedings{ISSTA14p437,
author = {René Just and Darioush Jalali and Michael D. Ernst},
title = {Defects4J: A Database of Existing Faults to Enable Controlled Testing Studies for Java Programs},
booktitle = {Proc.\ ISSTA},
publisher = {ACM},
pages = {437--440},
doi = {},
year = {2014},
}
A Format String Checker for Java
Konstantin Weitz, Siwakorn Srisakaokul, Gene Kim, and Michael D. Ernst
(University of Washington, USA)
Java supports format strings, but their use is error prone
because: Java’s type system does not find any but the most
trivial mistakes, Java’s format methods fail silently, and for-
mat methods are often executed infrequently.
This paper presents the Format String Checker that is
based on the format string type system presented in [3].
The Format String Checker guarantees that calls to Java’s
Formatter API will not throw exceptions.
We evaluate the Format String Checker on 6 large and
well-maintained open-source projects. Format string bugs
are common in practice (we found 104 bugs), and the an-
notation burden on the user of our type system is low (on
average, for every bug found, only 1.0 annotations need to
be written).
@InProceedings{ISSTA14p441,
author = {Konstantin Weitz and Siwakorn Srisakaokul and Gene Kim and Michael D. Ernst},
title = {A Format String Checker for Java},
booktitle = {Proc.\ ISSTA},
publisher = {ACM},
pages = {441--444},
doi = {},
year = {2014},
}
Info
Constructing Coding Duels in Pex4Fun and Code Hunt
Nikolai Tillmann, Jonathan de Halleux
,
Tao Xie , and Judith Bishop
(Microsoft Research, USA; University of Illinois at Urbana-Champaign, USA)
Pex is an automatic white-box test-generation tool for .NET. We have established that games can be built on top of Pex to open the tool to students and to the general public. In particular, we have released Pex4Fun (www.pexforfun.com) and its successor Code Hunt (www.codehunt.com) as web-based educational gaming environments for teaching and learning programming and software engineering. In Pex4Fun and Code Hunt, the main game type is a coding duel, where a player writes code in a method to achieve the same functionality as the secret method implementation, based on feedback provided by the underlying Pex tool. Players iteratively modify their code to match the functional behavior of the secret method. The scope of duels extends from the simplest one-line method to those including advanced concepts such as writing parameterized unit tests and code contracts. We have also used the game type for competitions with thousands of players, and have found that it differentiates well between beginners and top coders. This tool demonstration shows how coding duels in Pex4Fun and Code Hunt can be constructed and used in teaching and training programming and software engineering.
@InProceedings{ISSTA14p445,
author = {Nikolai Tillmann and Jonathan de Halleux and Tao Xie and Judith Bishop},
title = {Constructing Coding Duels in Pex4Fun and Code Hunt},
booktitle = {Proc.\ ISSTA},
publisher = {ACM},
pages = {445--448},
doi = {},
year = {2014},
}
proc time: 1.06