Research on pre-trained transformer models incorporating domain knowledge in government big data analysis

Key Points

• This research aims to explore how incorporating domain knowledge into pre-trained transformer models can improve government big data analysis.
• Analyzed existing pre-trained transformer models
• Integrated domain-specific knowledge into model architecture
• Evaluated performance metrics on big data analysis tasks
• Improved accuracy in data classification tasks
• Enhanced interpretability of model outputs
• Demonstrated potential for better decision-making in government data utilization

Abstract