Using a ViT to convert images of equations into LaTeX code

The goal of this project is to create a learning based system that takes an image of a math formula and returns corresponding LaTeX code. The model consist of a ViT [1] encoder with a ResNet backbone and a Transformer [2] decoder. We need paired data for the network to learn.