Metamorphic testing (MT) is a black-box testing technique to alleviate the oracle problem via leveraging Metamorphic Relations (MRs) based on the domain knowledge of the software under test. In recent years, software testing researchers have made significant research advancements on MT in fundamental theories (e.g., MR identification and composition), methodologies (e.g., test input generation), and fault detection effectiveness in various application domains. However, there still exist some major challenges of MT yet to be addressed, such as the needs for a general MR description language, and an automated system that supports all major steps of MT and integrates the various MT tasks. To address these challenges, we have developed a general MR description language (called the Category-Choice Metamorphic specification Language; abbreviated as CCML), through which an automated supporting tool (called the Category-Choice Metamorphic testing Tool; abbreviated as CCMT) integrating the various MT tasks has been built. CCMT supports the automatic generation of test inputs and composite MRs and integrates various run-time optimization strategies. We have also conducted empirical studies to evaluate the expressiveness of CCML and the performance of CCMT in various testing aspects. Overall, our empirical findings are encouraging, and have demonstrated the merits of CCML and CCMT. In this regard, our work contributes to improving the fault detection effectiveness, efficiency, and practicality of MT and, hence, brings the use of MT to a new height.
Sun et al. (Sat,) studied this question.