Development and validation of an artificial intelligence chatbot to assess personality

Recent technological advances have allowed researchers to apply automated, language-based machine learning models as alternatives to self-reports for assessing personality. However, previous work has largely overlooked the multidimensional nature of personality and lacked in-depth exploration of validity issues. In this paper, we examined novel methods for leveraging artificial intelligence (AI), natural language processing (NLP), machine learning, and automation to systematically glean personality-related information from textual data which offers rich information and reflects various aspects of personality but has been severely underutilized. We connected the five-factor (or Big Five) model (comprised of openness to new experiences, conscientiousness, extraversion, agreeableness, and neuroticism) with NLP from two angles: 1) a psychometric and construct validity perspective (i.e., the degree to which information extracted from textual data reflects personality constructs), and 2) an applicability perspective (i.e., the ability to elicit personality-relevant information from text in line with psychological and organizational principles). Innovatively, we built an interactive tool to automatically and adaptively prompt for, collect, and analyze personality-relevant topic-based (i.e., honoring the Big Five factorial structure) narrative data through conversations conducted by an AI chatbot. Results showed significant improvements in various validities of the new personality assessment tool compared to existing applications. Potential reasons for the improvement magnitudes, limitations of the current methods, and future directions are discussed.