Multimodal Translation Method Using Summary-Level Image Utilization

Key Points

• MVNMT effectively reduces excessive translation issues compared to prior models using token-level image utilization.
• The model incorporates summary-level images to better represent contextual information, enhancing translation fidelity.
• Using a variational autoencoder, MVNMT extracts a common latent representation from both text and images.
• Evaluations demonstrate that MVNMT surpasses traditional text-only translation models in performance metrics.

Abstract