Problem-Related Performance Metrics of Deep Learning Models: Application to Swimmer Pose Estimation in Underwater Environments

• Scale-aware evaluation framework for underwater swimmer pose estimation • Continuous, tolerance-NLA for anatomical keypoints • Analysis of core-distal performance gradients in underwater imagery Accurate 2D swimmer pose estimation in underwater environments remains a challenging task due to optical distortions, dynamic occlusions, and the highly multi-scale nature of anatomical landmarks. Conventional evaluation metrics adopted from generic computer vision benchmarks are often inadequate to characterise the functional reliability required in sports biomechanics, particularly when small and fast-moving joints are involved. This work proposes a problem-orientated evaluation framework for underwater swimmer pose estimation, applied to seven deep learning model configurations differing in training data composition, pre-processing strategies, and parameter optimisation. Beyond the indicators based on standard confusion-matrix, a dual assessment strategy is introduced, combining strict anatomical tolerance thresholds with a continuous tolerance-Normalised Localisation Accuracy (NLA). Keypoint-specific tolerances are derived from the spatial extent of each anatomical region, allowing scale-consistent evaluation throughout the kinematic chain. Experimental results show a pronounced performance gradient from core body segments to distal extremities, highlighting the limitations of binary metrics for small joints. Models trained on heterogeneous raw datasets achieve the best overall performance (Global Performance Index = 78.69), demonstrating superior robustness and generalisation. Comparative analysis reveals that binary tolerance-based metrics are overly punitive for distal landmarks and tend to obscure the true localisation capability of the models. The proposed continuous NLA provides a more informative representation of spatial uncertainty and measurement quality. These findings emphasise the importance of problem-related, scale-aware evaluation metrics and confirm data diversity as a more effective driver of robustness than aggressive pre-processing in underwater swimmer pose estimation.