Authors:
Łukasz Maziarka, Tomasz Danel, Sławomir Mucha, Krzysztof Rataj, Stanisław Jastrzębski
Abstract:
Properties of a molecule depend on a variety of relationships between its atoms. On a high level, these relationships might include spatial proximity, the existence of a chemical bond, or simply a co-occurrence of two atoms. However, the commonly used graph-based models use only the chemical bonds to define the neighbourhood. Motivated by this we propose Molecule Transformer (MT) model. Our key innovation is augmenting the attention mechanism in Transformer using the inter-atomic distances, and the molecular graph structure. Experiments on molecular property prediction tasks show that our method outperforms all the other tested models on multiple tasks. We also show that individual attention heads implement different yet chemically interpretable functions.
