Building an Adaptive Vietnamese License Plate Recognition and Retrieval System using Multi-Task Deep Learning

Automatic License Plate Recognition (ALPR) is an essential component of intelligent transportation, yet its performance is often significantly degraded by real-world image distortions and regional plate format complexities. This research addresses these challenges by proposing a highly adaptive, multi-task deep learning framework specifically designed for the Vietnamese license plate context. The system targets the unique diversity of Vietnamese plates while robustly handling low-quality image inputs through a multi-stage, conditional pipeline. First, a real-time object detection model localizes all license plate instances. The core component is a lightweight Quality Assessment Module (QAM), which acts as an intelligent router, classifying each plate as “clear”, “restorable”, or “unrestorable”. Based on this assessment, “restorable” images are selectively forwarded to a Swin2SR-based restoration network, while “clear” images bypass this step to optimize throughput, and “unrestorable” inputs are discarded. Finally, a Transformer-based Optical Character Recognition (OCR) model transcribes the characters to retrieve vehicle information. Experimental evaluations confirm the effectiveness of this end-to-end adaptive strategy, achieving a Character Accuracy of 96.07% and a Sequence Accuracy of 89.69%. These results demonstrate significant robustness against real-world distortions compared to traditional monolithic pipelines, offering a practical and efficient solution for ALPR applications in Vietnam.