Paper

Generating Structured Test Data with Specific Properties using Nested Monte-Carlo Search

by S. Poulding and R. Feldt

Abstract

Software acting on complex data structures can be challenging to test: it is difficult to generate diverse test data that satisfies structural constraints while simultaneously exhibiting properties, such as a particular size, that the test engineer believes will be effective in detecting faults. In our previous work we introduced GödelTest, a framework for generating such data structures using non-deterministic programs, and combined it with Differential Evolution to optimize the generation process.

Monte-Carlo Tree Search (MCTS) is a search technique that has shown great success in playing games that can be represented as sequence of decisions. In this paper we apply Nested Monte-Carlo Search, a single-player variant of MCTS, to the sequence of decisions made by the generating programs used by GödelTest, and show that this combination can efficiently generate random data structures which exhibit the specific properties that the test engineer requires. We compare the results to Boltzmann sampling, an analytical approach to generating random combinatorial data structures.

The work that went in to this paper was partly funded by The Knowledge Foundation (KKS) through the project 20130085 Testing of Critical System Characteristics (TOCSYC).

Bibtex

@InProceedings{poulding2014nmcsGT,
  author =    "Simon Poulding and Robert Feldt",
  title =     "Generating Structured Test Data with Specific Properties using Nested Monte-Carlo Search",
  booktitle = "Proceedings of the Genetic and Evolutionary Computation Conference (GECCO)",
  year =      "2014",
  pages =     "",
  publisher = "IEEE",
  keywords = "Search-based software testing, Automated testing",
}