Providing projective and affine invariance for recognition by Multi-Angle-Scale Vision Transformer

Abstract Deformed 2D shape recognition finds applications in many unrelated areas, such as marketing, OCR, and autonomous vehicles. An enormous effort has been devoted to this in the literature, based on direct geometric approaches, although with limited results or performance. More recently, many machine-learning approaches have been proposed with satisfactory results only when the deformation is a weak affine at best. This paper introduces MASViT, a deep-learning-based solution that outperforms state of the art methods in the recognition of affinely and projectively deformed images. A crucial point in our setting is the absence of deformed images during training phase. Our approach employs 1D convolutional filters corresponding to straight lines crossing the shape in the polar domain, preserving collinearity, a basic projective invariant. Angular sequences deriving from the polar domain integrate well with the ViT architecture, as these patch embeddings are geometrically coherent, enhancing suitability for the transformer encoder. We also introduce several regularization techniques to boost the generalizability of model. To validate the approach, we curated new test datasets derived from the GTSRB dataset (traffic signs). Through extensive experiments, we demonstrate that this approach surpasses state-of-the-art models, particularly when dealing with images subjected to severe affine and projective deformations.