Knowledge distillation improved RNN model AUROC from 0.9186 to 0.9208 at 61.45% parameter retention on ECG multi-label classification.
Do artificial neural network compression techniques maintain multi-label ECG classification performance while reducing model size and inference time compared to uncompressed models?
Neural network compression techniques like pruning and knowledge distillation can significantly reduce the size of ECG classification models, sometimes even improving performance (as seen in RNNs), enabling deployment on resource-constrained edge devices.
Estimación del efecto: Absolute AUROC increase of 0.0022
Tasa de eventos absoluta: 0.9208% vs 0.9186%
Proliferation of wearable healthcare devices has created the need to deliver artificial intelligence applications for these resource-constrained devices to achieve faster, localized decision-making, by bringing computation closer to the data sources, for improved responsiveness and privacy. This contribution presents the results of an experimental evaluation of artificial neural network compression techniques, including quantization, structured pruning, and knowledge distillation, applied to multi-label classification of electrocardiogram (ECG) signals. The experiments were carried out on the PTB-XL dataset using three deep learning models, i.e. an LSTM-based recurrent neural network, a 1D convolutional neural network, and a 1D residual neural network. The results show how the compression methods impact model quality and highlight opportunities to reduce model size and accelerate inference, thereby enabling effective deployment on resource-constrained, edge devices.
Hołda et al. (Thu,) conducted a other in ECG multi-label classification (n=18,885). Neural network model compression techniques (quantization, structured pruning, knowledge distillation) vs. Uncompressed neural network models was evaluated on Multi-label classification AUROC (macro-averaged) (Absolute AUROC increase of 0.0022). Knowledge distillation improved RNN model AUROC from 0.9186 to 0.9208 at 61.45% parameter retention on ECG multi-label classification.