Cost-Sensitive Threshold Optimization for Network Intrusion Detection: A Per-Class Approach with XGBoost

Machine learning-based Network Intrusion Detection Systems (NIDSs) typically optimize uniform metrics such as accuracy and F1-score, overlooking the asymmetric cost structure of real-world security operations, where a missed attack (False Negative (FN)) far outweighs a false alarm (False Positive (FP)). We propose a cost-sensitive threshold optimization framework based on XGBoost, using a 10:1 FN-to-FP cost ratio derived from established cost models. We first demonstrate that the default threshold of 0.5 is suboptimal and that a globally optimized threshold of 0.08 substantially reduces total cost. However, a single global threshold cannot accommodate the heterogeneous detection characteristics of diverse attack types. We therefore introduce Per-Class Thresholding, which assigns independently optimized thresholds to each attack class. Evaluated on CIC-IDS2018 and UNSW-NB15 across five independent random seeds, our method achieves a 28.19% cost reduction over the Random Forest baseline on CIC-IDS2018, demonstrating that attack classes undetectable under the global threshold—including DDoS attack-LOIC-UDP (100%), DoS attacks-SlowHTTPTest (99.79%), and FTP-BruteForce (98.16%)—can achieve near-complete cost elimination through individual per-class threshold search. Cross-dataset validation on UNSW-NB15 further confirms that per-class thresholding consistently improves class-level detection, with cost reductions of 74.10% for Reconnaissance, 69.06% for Backdoor, and 54.42% for Analysis attacks. These results confirm that class-specific threshold calibration is essential for cost-effective intrusion detection.