LLM predicts human behavior: A BERT-based approach for conscientiousness personality trait detection from online content

Advancements in deep learning have significantly transformed the Natural Language Processing (NLP) domain, enabling machines to understand and analyze human language with remarkable accuracy. One of the emerging applications of NLP is personality prediction, where AI-driven models process textual data to uncover underlying behavioral traits. With the rise of the user-generated content on digital platforms, individuals continuously express their thoughts, opinions, and emotions, offering a valuable source for personality assessment. Personality is a fundamental aspect of human behavior, reflecting human cognition and social patterns. With the advent of social media, understanding and predicting personality traits have become a trending topic due to its significance because of huge applications in diverse areas like psychology, health care, marketing, education, etc. Automatic recognition of human personality traits from social media content provides deeper insights into the users' behaviors. In this study, our aim is to predict personality of conscientiousness trait which is crucial for understanding whether a person is a judge or perceiver for analyzing the individuals' perception as organized, dependable, and goal-oriented behavior. For empirical analysis, we investigated MBTI dataset with state-of-the-art large language model BERT reveals the highest accuracy of 97% as compared to different Machine Learning (ML) and Deep Learning (DL) integrate with feature engineering techniques to extract information including textual features with ML, as well as deep features integrate with DL. Comprehensive results analysis reveals BERT as a strong predictor for personality detection from relevant literature, offering valuable insights into user behavior on social channels. • Achieving 97% accuracy in MBTI personality prediction using BERT-Large architecture. • BERT outperforms traditional ML, DL, and ensemble methods in personality inference. • Textual features and deep embeddings reveal how LLM captures language cues. • Identifying key linguistics behind BERT predictions using SHAP and LIME analyses.