Imagine yourself designing a web page on your IPad using an Apple Pencil, and within seconds based on the design that you have made your web page is created! Well, that seems to be a moon shot. However, researchers have started taking steps towards this phantasmagorical dream. OpenAI, in their requests for research, has put down a project that I feel could be considered as the first ‘baby’ step towards it.

Im2latex. The project aims at creating an Deep Learning model that could help us in converting the image of a mathematical formula into its corresponding Latex form! Well, that sounds like a great idea in itself, isn’t it? No more messing around when writing reports and papers, and who knows if we could extend this idea further to write complete reports. I feel this idea has a potential to become a comprehensive product in iteslf, to convert what is ‘visually portrayed’ in an image into a code, whether it be Latex, HTML so on and so forth.

So looking at the prospects of the project we started working on this idea, to see whether some approaches used for language translation may be applied in this domain! Primarily what we are trying to accomplish, is a replica of the model based on simple convolutions in Convolutional Sequence to Sequence Learning, that Facebook AI Research has proposed for English to German translation. We believe that Convolutional features have sufficient information, and it’s a worth a shot to try to work around a possible method based on pure convolutions to accomplish this task!

We might fail. However, the quest is towards understanding the intricacies of Deep Learning and Neural Machine Translation as a whole.

We would be continuously posting the ideas and the experiments that we are trying. So if anyone has any suggestions or would love to join us can email us, and we would be more than happy to discuss our plans and ideas to create a synergy!