Fault Detection Effectiveness of Source Test Case Generation Strategies for Metamorphic Testing.

Metamorphic testing is a well known approach to tackle the oracle problem insoftware testing. This technique requires the use of source test cases thatserve as seeds for the generation of follow-up test cases. Systematic design oftest cases is crucial for the test quality. Thus, source test case generationstrategy can make a big impact on the fault detection effectiveness ofmetamorphic testing. Most of the previous studies on metamorphic testing haveused either random test data or existing test cases as source test cases. Therehas been limited research done on systematic source test case generation formetamorphic testing. This paper provides a comprehensive evaluation on theimpact of source test case generation techniques on the fault findingeffectiveness of metamorphic testing. We evaluated the effectiveness of linecoverage, branch coverage, weak mutation and random test generation strategiesfor source test case generation. The experiments are conducted with 77 methodsfrom 4 open source code repositories. Our results show that by systematicallycreating source test cases, we can significantly increase the fault findingeffectiveness of metamorphic testing. Further, in this paper we introduce asimple metamorphic testing tool called "METtester" that we use to conductmetamorphic testing on these methods.