Model-based clustering of multiple images incorporating covariates

In this paper, we develop a novel method for clustering multiple images while adjusting for the effect of available covariates on cluster membership. The key strategy is to represent each image as two-dimensional functional data and formulate a functional latent class mixed model, which fully leverages the structural information of images while effectively addressing their high dimensionality and accounting for covariate effects. We apply the proposed method to color intensity matrices extracted from patient-sourced smartphone fingernail photos to identify distinct subgroups, while adjusting for the effect of image metadata, which may act as an effect modifier on cluster membership. Information on these subgroups can assist public health officials in low-resource settings by enabling rapid and non-invasive identification of high-risk subpopulations for anemia, thereby facilitating the timely delivery of targeted interventions. The results suggest that the three clusters identified by the proposed method correspond to varying levels of anemia risk, with 0%, 79%, and 86% of subjects in each cluster classified as anemic. These findings demonstrate the utility of the proposed method and highlight the potential of a smartphone application leveraging fingernail images for non-invasive and cost-effective anemia screening.