Comparison of Artificial Intelligence Tools With Human Coding for Sentiment, Topic, and Thematic Analysis Tasks of Public Health Datasets During the COVID-19 Pandemic in Australia: Case Study

Key Points

• This research aims to evaluate the effectiveness of AI tools compared to human coding in analyzing public health datasets during the COVID-19 pandemic.
• Compared AI tools and human coding for sentiment, topic, and thematic analysis in health-related texts.
• Utilized public health datasets from the COVID-19 pandemic in Australia.
• Assessed reliability of AI tools combined with human oversight for better accuracy.
• AI tools showed limited reliability for sentiment classification without human input.
• Thematic exploration was more promising when AI was used alongside human review.
• Further applications and improvements are necessary for analyzing non-English datasets.

Abstract