Federated learning (FL) facilitates multiple enterprises to jointly train a powerful global model for machine fault diagnosis, while preserving the data privacy in each enterprise. However, data heterogeneity arising from varying operating conditions, limited local data, and imbalanced fault classes across different enterprises as clients, inevitably hinders the optimization of the global model, often leading to poor generalization. To address this issue, we propose a novel FL paradigm, Fed-FM, which leverages powerful foundation models (FMs) for multimodal learning to enhance the generalization of the global model in heterogeneous FL. Fed-FM utilizes the pre-trained FM for textual feature learning and contrastively trains the local models with the FMs to expand the knowledge of the local models, thereby improving the generalization of the global model in FL. In addition, an L2-norm regularization term is explored on the local data features to align its marginal distributions, further reducing the domain distribution shift within and across clients. Through the multimodal feature learning and feature alignment, the aggregated global model can be enhanced with strong generalization. Experiments performed on three fault datasets under various FL settings indicate that our proposed Fed-FM improves the generalization of the global model with significant superiority, achieving performance gains compared with other FL methods.
Sun et al. (Sun,) studied this question.