Erudit AI SaaS: an artificial intelligence tool based on RoBERTa for classifying employee engagement

Engagement has been defined since its inception by several currents, mainly two different ones such as those of Kahn (1990) and Schaufeli (2002). Based on these two main currents, self-report instruments for its measurement have been increasing over time. However, in addition to the limitations of the instrument itself (low reliability, low use, etc.), these instruments also have limitations inherent to self-report measures in general (social desirability, correction time, etc.). To solve this problem, and taking into account that the way of communicating through corporate tools in companies (slack, Gmail, Microsoft teams, etc.,) has increased considerably, this study proposes the validation of an artificial intelligence tool (Erudit AI SaaS) that allows to know the level of engagement of employees by analyzing the text they write in their corporate tools. To test whether the tool is really assessing the level of engagement we compared the self-report instruments UWES-9 and JRA with the Erudit AI SaaS. Thirty-five employees were selected as participants for the study and the instruments were administered on five different occasions. To determine convergent and divergent validity, bivariate correlations were performed, in the case of reliability, this was calculated using intraclass correlation indexes. The results showed good results for convergent validity between the Erudit AI SaaS and the self-report instruments UWES-9 and JRA. Divergent validity also yielded good results between the Erudit AI SaaS and MBI. The reliability, however, was moderate. These findings provide preliminary evidence supporting the construct validity of Erudit AI SaaS for estimating employee engagement through textual data extracted from corporate communication platforms. However, while initial validity and reliability are encouraging, further research with larger samples and more robust statistical analyses is needed to generalize the results and strengthen the tool’s psychometric foundation.