Integrating information system success and task technology models to develop a fit artificial intelligence-based evaluation system

Purpose Artificial intelligence (AI) has been increasingly applied in various learning activities, both fully online and blended. However, its integration into evaluation systems remains limited. This study aims to address this gap by identifying development factors through the integration of the Information System (IS) Success Model and the Task–Technology Fit (TTF) Model to design a fit AI-based evaluation system. Design/methodology/approach A quantitative survey method was used. Data were collected through a structured online questionnaire administered to 940 students from two state universities in Indonesia. The data were analyzed using descriptive statistics and partial least squares structural equation modeling to test validity, reliability and the relationships between constructs. Findings This study proposes a model for developing an AI-based evaluation system, demonstrating that IS Success and TTF factors jointly determine utilization, performance impact and net benefit. The results of this study show that this integrated model provides valuable recommendations regarding the contribution of service quality and task–technology alignment, as it enables optimal benefits from the system. Practical implications The findings of this study have direct implications for online and blended learning in higher education. Integrating an AI-based evaluation system into a learning management system can serve as an early warning tool, as it offers timely feedback on students’ learning outcomes. Originality/value This research provides a novel theoretical-empirical perspective by integrating the IS Success Model and the TTF Model to specify design requirements for an AI-based evaluation system. Rather than developing an AI tool, this study identifies the system, task-fit and quality factors necessary to guide future AI-driven assessment implementation.