Hybrid Usability Evaluation of an Automotive REM Tool: Human and LLM-Based Heuristic Assessment of IBM Doors Next

Requirements Engineering and Management (REM) tools play a significant role in ensuring project compliance and efficiency. Automotive engineering must comply with regulatory standards, requiring detailed documentation, rigorous testing, and solid traceability. Despite their importance, REM tools are underexplored from the usability and user experience perspective (UX), even though poor usability can hinder development workflows across stakeholder teams. This study presents a case study of heuristic usability evaluation of IBM DOORS Next Generation, conducted with expert evaluators, using Nielsen’s 10 Usability Heuristics as an evaluation framework. The identified issues were analyzed in terms of impacted heuristics and severity ratings. Additionally, we underwent a Large Language Model (LLM)-based heuristic evaluation, using ChatGPT-5, prompted with the same heuristic set and static screenshots. The LLM detected several issues overlapping with human findings (32%), as well as new ones (23%); therefore, 55% of its outputs are considered valid and 45% are unconfirmed. This highlights both the potential and limitations of AI-driven usability assessment. Overall, the findings underscore the usability challenges of REM tools and suggest that LLMs may serve as complementary evaluators, accelerating early-stage heuristic inspections in safety-critical engineering environments.