PulseExploreJournal ClubDebatesTrendingResearchersJournals
Instagram
HomeExploreJournal ClubTrending
Synapse
⌘+K
Synapse
March 22, 2026Digital humanities quarterly0 citationsOpen Access

Unveiling the Critical Nexus of Data Preprocessing and Transparent Documentation for Result Quality and Reproducibility in Digital History

View Full Paper
CSClodomir SantanaDFDV FilhoMBMichał Bojanowski

Key Points

  • To highlight how data preprocessing and documentation affect research quality and reproducibility in digital history.
  • Evaluated data cleaning and preprocessing effects on model performance.
  • Conducted a comparative analysis between LDA, LSI, and GSDMM algorithms.
  • Presented a topic modelling case study using 160,000 records from the Atlantic Portuguese Empire.
  • Found that proper data preprocessing enhances model performance.
  • Illustrated the significance of transparent documentation for research reproducibility.
  • Showed topic modelling's potential in digital historical studies.

Abstract

This study underscores the importance of adequate data preprocessing, transparency, and documentation in digital history research, showcasing how these often overlooked practices impact research quality and reproducibility. We present a topic modelling case study involving over 160,000 records of official correspondence of the Atlantic Portuguese Empire from 1640 to 1822 to illustrate how these practices, associated with standardised formats and metadata conventions, facilitate the sharing and reproduction of experiments. First, we evaluate the impact of data cleaning and preprocessing on model performance. Second, concerning model selection, we compare the performance of latent Dirichlet allocation (LDA), latent semantic indexing (LSI), and Gibbs sampling algorithm for a Dirichlet mixture model (GSDMM). Besides stressing the underestimated significance of data preprocessing and transparent documentation to strengthen research robustness and contribute to a reproducibility culture, we also demonstrate the potential of topic modelling in digital historical studies, specifically in the context of the Atlantic Portuguese Empire.

Ask AI
Helpful
Bookmark
Share
View Full Paper

Cite This Study

Santana et al. (2025) studied this question.

synapsesocial.com/papers/69bf390ac7b3c90b18b43454https://doi.org/10.63744/bgd67ykve8v7
Ask AI
Helpful
Bookmark
Share
View Full Paper