FURINA: A Multi-Agent Framework for Automated Patch Correctness Assessment

Automated Program Repair (APR) has made substantial advancements in fixing real-world defects. However, the overfitting problem, namely the phenomenon where APR-generated patches pass the given test suite but fail to correctly fix the underlying defects, remains a major challenge. To mitigate this problem, numerous Automated Patch Correctness Assessment (APCA) methods have been proposed. Among them, learning-based APCA methods currently represent the state-of-the-art (SOTA). However, these methods require training on large, high-quality patch datasets. Moreover, their lack of interpretability, as they often produce only simple binary predictions, provides little support for subsequent manual review. Finally, existing APCA methods often fail to fully leverage the rich contextual information both within and beyond the patched program, and they struggle to incorporate expert knowledge about patch correctness assessment. Large Language Models (LLMs) offer a promising path to address these limitations. Trained on vast corpora, they demonstrate strong generalization across tasks and an ability to learn from experience, allowing them to provide interpretable explanations in a manner analogous to human reasoning. In this paper, we propose a multi-agent framework FURINA, which formulates APCA as a reasoning task. The framework employs a pre-trained retriever and four specialized agents to analyze a candidate patch from multiple perspectives: a Profiler to analyze developer intent from failing tests and bug reports; an Interpreter to analyze code semantics according to the function-level patch; a Critic to analyze potential degradations from the patch context; and a final Judge to synthesize the patches, failing tests, bug reports, intermediate reasoning reports generated by other agents, the retrieved similar patches, along with instructive empirical tips derived from prior APCA studies to determine patch correctness. Evaluation results on the public benchmark CACHE-small demonstrate that FURINA, when equipped with popular LLMs, outperforms SOTA APCA baselines, achieving improvements of at least 5.58% in \(Accuracy\) and 5.93% in \(F1\) . To further assess its reasoning quality, we design a new evaluation scheme that considers three dimensions: factual consistency, logical clarity, and readability. The results show that FURINA achieves a score of at least 4.8 out of 5, demonstrating superior reasoning quality compared with SOTA baselines. Finally, based on our constructed dataset, FURINA still achieves competitive performance, indicating strong generalization ability.