Transformer Models for Protein Contact Map Prediction

Introduction

Inter-residue contact maps constrain protein folding, and accurate prediction from sequence alone remains valuable for proteins lacking close structural homologs.

Method

We apply an axial-attention transformer over multiple sequence alignment columns, supervised on contacts derived from known structures.

Results

For orphan proteins the model improves long-range contact precision by 11 points over a coevolution baseline.