Deep reinforcement learning for network resource optimization in MIMO-NOMA networks to maximize utilization with minimal overhead

The next-generation multiple-input Multiple-Output Non-Orthogonal Medium Access (MIMO-NOMA) system requires seamless mobility, enhanced spectral efficiency, and higher sum rates with minimal interference. Selecting the optimal network and optimizing resources to meet user quality-of-service (QoS) requirements is challenging in highly crowded, fast-fading MIMO-NOMA networks with high mobility, resource fluctuations, and interference. Various network selection and resource optimization models have been designed using predictive machine learning (ML) and deep learning (DL) techniques with good results. However, in the rapidly fading MIMO-NOMA system, existing methods fail to optimize both network selection and resource allocation. This study introduces the Optimal Spectral Interference Aware Network Resource Optimization (OSIANRO) strategy for the MIMO-NOMA system. The OSIANRO strategy introduces effective network selection optimization using an enhanced Extreme Gradient Boosting (XGB) model with an ideal feature identification mechanism to reduce network selection failures. Then, the OSIANRO strategy leverages effective resource optimization to improve spectral efficiency by increasing the sum rate while minimizing interference. Finally, optimal performance is achieved by leveraging a deep reinforcement learning (DRL) model to optimize network resources. The simulation study shows the proposed model reduces collisions by 53.9%, increases the sum rate by 18.76%, and enhances spectral efficiency by 25.55% compared to baseline models under urban and expressway propagation models.