This paper proposes a new spectrum sensing technique for Orthogonal Time Frequency Space (OTFS) waveforms using Deep Q-Network (DQN) reinforcement learning (RL) to improve the spectrum efficiency of Beyond 5G (B5G) radio systems for vehicular communication. OTFS benefits high-mobility scenarios by providing strong immunity against Doppler effects and delay spread. The proposed DQN-based method optimizes the decision-making related to sensing by learning from environmental interactions and real-time adjustment of actions to maximize spectrum utilization. RL improves detection accuracy, even when the channel conditions are poor. Numerical simulations demonstrate the effectiveness of the proposed method. The Probability of detection (Pd) improves significantly with increasing signal-to-noise ratio (SNR), achieving Pd =0.98 at SNR =-10 dB while maintaining a low probability of false alarm of 0.05 and 0.02 for SNR = -5 dB and -10 dB. The Pd vs. Pfa curve confirms superior detection performance with a reduced error trade-off compared to conventional approaches such as Recurrent Neural Network (RNN), convolutional Neural Network (CNN), cyclostationary spectrum (CS), matched filter (MF) and Energy detection (ED). Furthermore, the SNR vs. Bit Error Rate (BER) analysis shows a substantial reduction in BER, reaching 10 -3 at SNR=4 dB, highlighting enhanced communication reliability. Power spectral density (PSD) analysis validates efficient spectrum usage with minimal interference. The proposed DQN method, based on deep RL, ensures high spectrum efficiency, high detection accuracy, and reliable performance in complex wireless environments, paving the way for robust and adaptive B5G radio communication systems.
Kumar et al. (Mon,) studied this question.