IDP-Bert: Predicting Properties of Intrinsically Disordered Proteins (IDP) Using Large Language Models

Abstract

Intrinsically Disordered Proteins (IDPs) constitute a large and structure-less class of proteins with significant functions. The existence of IDPs challenges the conventional notion that the biological functions of proteins rely on their three-dimensional structures. Despite lacking well-defined spatial arrangements, they exhibit diverse biological functions, influencing cellular processes and shedding light on the disease mechanisms. However, it is expensive to run experiments or simulations to characterize this class of proteins. Consequently, we designed an ML model that relies solely on amino acid sequences. In this study, we introduce IDP-Bert model, a deep-learning architecture leveraging Transformers and Protein Language Models (PLMs) to map sequences directly to IDPs properties. Our experiments demonstrate accurate predictions of IDPs properties, including Radius of Gyration, end-to-end Decorrelation Time, and Heat Capacity.