Providing Projective and Affine Invariance for Recognition by Multi-Angle-Scale Vision Transformer

Abstract

The recognition of deformed planar shapes finds applications in many unrelated areas, such as marketing, OCR, and autonomous vehicles. An enormous effort has been devoted to this in the literature, based on direct geometric approaches, although with limited results or performance. More recently, many machine learning approaches have been pro posed with satisfactory results only when the deformation is a weak affine at best. This thesis introduces the Multi-Angle-Scale Vision Transformer, MASViT, a deep-learning based solution that outperforms state of the art methods in the recognition of affinely and projectively deformed images. A crucial point in our setting is the absence of deformed images during training phase. Our approach employs 1D convolutional filters correspond ing to straight lines crossing the shape in the polar domain, preserving collinearity, a basic projective invariant. Angular sequences deriving from the polar domain integrate well with the Vision Transformer (ViT) architecture, as these patch embeddings are geometrically coherent, enhancing suitability for the transformer encoder. We also introduce several reg ularization techniques to boost the generalizability of model. To validate the approach, we curated new test datasets derived from the German Traffic Sign Recognition Benchmark (GTSRB). Through extensive experiments, we demonstrate that this approach surpasses state-of-the-art models, particularly when dealing with images subjected to severe affine and projective deformations.