Investigating Configurations for Automated Crash Reproduction in JavaScript

Automated Crash Reproduction is the process of generating a test case which reproduces a software crash. The process usually identifies a crash as reproduced when a generated test can match an error with a similar execution context to the original crash, with most approaches aiming to match a stack trace. We present research outcomes which deepen the understanding of this area. The first is a reproduction study of the existing tool EvoCrash, providing insight into areas of Automated Crash Reproduction which require further research. The second is CrashJS: a benchmark dataset for Automated Crash Reproduction for NodeJS programs.

CrashJS contains 453 crashes from several pre-existing sources such as benchmarks for JavaScript bugs and security vulnerabilities, GitHub issues, and generated crashes from existing JavaScript test generation tools.

We present our changes to the SynTest-JavaScript test generation tool to leverage the existing framework for Automated Crash Reproduction. This resulted in the first tool for Automated Crash Reproduction for JavaScript programs. Our tool is the second for dynamically-typed languages, with most existing research focused on statically-typed languages. We present an empirical evaluation of fitness functions for Automated Crash Reproduction for NodeJS programs. We found fitness functions which can outperform the existing state of the art fitness functions for Automated Crash Reproduction, reproducing 18.9% more crashes from CrashJS, or 28.9% more crashes, if partial reproductions are included. Finally, we applied our fitness functions in EvoCrash to assess their suitability in Java. We found that our fitness functions could not reproduce more crashes than using the EvoCrash fitness function. However, we discovered our fitness functions could reproduce crashes faster than when using the EvoCrash function.