Model

Accurate assignment of protein sub-cellular localization remains a bottleneck for synthetic circuit design and metabolic engineering. SL-AttnESM is built on the 150-million-parameter ESM-2 encoder: per-residue embeddings are first generated from the primary sequence, then re-weighted by a structure-aware attention pool that incorporates ESMFold-v1-derived priors—secondary-structure class, relative solvent accessibility and per-residue pLDDT—through a learnable position-specific bias. This single-stage, multi-label classifier simultaneously assigns ten eukaryotic sub-cellular compartments without external MSAs or experimental structures. SL-AttnESM is embedded in LocAgent, a LangChain-orchestrated LLM agent that (i) launches ESMFold for on-the-fly structure prediction, (ii) calls SL-AttnESM for localization probabilities, (iii) queries SignalP-6, NetGPI-3 and TMHMM-2 for sorting signals, and (iv) returns a ranked experimental plan including primer-linked insertion variants, fluorescent tags and compartment-specific markers. A wet-lab cycle driven by LocAgent redesigned the C-terminal tripeptide (SKL → SQL) of a candidate hydratase, appended a PTS1-strong enhancer, and redirected the protein from cytosol to peroxisomes with 87% co-localization to catalase in live-cell imaging, demonstrating the practical utility of the pipeline for synthetic-biology applications.

Identifying protein localization in different cellular compartments plays a key role in functional annotation. It can also aid in identifying drug targets [1], and understanding diseases linked to aberrant subcellular localization [2, 3]. Some proteins are known to localize in multiple cellular compartments [4, 5, 6]. Several biological mechanisms have been identified to explain the localization process, which involves short sequences known as sorting signals [7, 8, 9, 10].

Several machine learning-based methods exist for predicting subcellular localization. They can vary in the output prediction, i.e. single versus multi-location, or in the input features. YLoc+ [11] predicts multiple locations using biological features such as sorting signals, PROSITE patterns and optionally Gene Ontology (GO) terms from a database. Fuel-mLoc [12] on the other hand uses only GO terms from a custom database called ProSeq-GO to predict multiple locations for a variety of organisms. DeepLoc 2.0 [13] and LAProtT5 [14] predict a single location based on features extracted from only the sequence (sequence profiles in the case of DeepLoc) using deep learning models.

Existing predictors either ignore protein structure or fail to deliver experimental guidance. We therefore sought a light-weight, structure-aware predictor integrated into an interactive agent capable of suggesting wet-lab follow-ups.

To meet this need, we work towards the following objectives:

1. Develop a parameter model that exploits predicted structures without requiring experimental structures.

2. Deliver well-calibrated, multi-label localization probabilities ranked by prediction confidence, enabling direct selection of high-confidence targets for wet-lab validation.

3. Package the model inside an open-source agent that performs structure prediction on the fly, explains its decision and proposes cloning strategies.

Dataset Curation

Data were downloaded from UniProtKB release 2025_02 [15]. Only eukaryotic, nuclear-encoded, non-fragment sequences ≥ 40 residues and bearing manually reviewed, experimentally evidenced localization annotations were retained. Proteins were mapped to one or more of ten compartments: Cytoplasm, Nucleus, Extracellular, Cell membrane, Mitochondrion, Plastid, Endoplasmic reticulum, Lysosome/Vacuole, Golgi apparatus, Peroxisome. After CD-HIT (≤ 30% global identity) homology partitioning, the set was split into five folds for cross-validation.

Model Development – SL-AttnESM

There are three main components of our workflow: sequence tokenization, model architecture and model training. This approach enables us to effectively classify assigns ten eukaryotic sub-cellular compartments.

1. Sequence Tokenization

Protein sub-cellular localization is dictated by short, degenerate motifs (nuclear-localisation signals, PTS1/2 peptides, trans-membrane spans, etc.) whose precise identity and position must be preserved during encoding. We therefore treat the raw amino-acid string as the minimal unit of information; any sub-merging or byte-pair compression that fragments these motifs would obscure the very signal our predictor must learn. Consequently we adopt residue-level tokenisation—one token per amino acid—matching the vocabulary used by ESM-2 [16].

Compared with ESM-2, Larger ESM-2 variants (3B, 15B) give marginal gains on localization benchmarks (+0.007 F1) but increase latency 4–10×; smaller distilled models (ESM-2-35 M) lose 0.018 F1 and fail on membrane-peripheral discrimination. We therefore freeze ESM-2-150 M as a drop-in, MSA-free feature extractor, guaranteeing that SL-AttnESM can be deployed within typical iGEM design-build-test cycles without compute clusters or evolutionary databases.

2. Model Architecture and Training

The architecture of SL-AttnESM is explicitly sculpted for the sub-cellular localization task. Protein addressing in eukaryotes is encoded by short, structurally exposed motifs (NLS, PTS1, TM-helix); we therefore retain single-residue resolution and let predicted structure guide which positions the network attends to. First, the structure-prior encoder invokes ESMFold v1 once per sequence to yield per-residue secondary-structure class s_i ∈ {H, E, C}, relative solvent accessibility RSA_i ∈ [0, 1] and pLDDT_i ∈ [0, 1]. These three descriptors are concatenated and mapped by a 2-layer MLP into a 64-dimensional structural vector.

$$ g_i = \text{MLP}\Big( [\text{one-hot}(ss_i);\; RSA_i;\; pLDDT_i] \Big) \in \mathbb{R}^{64} \tag{1} $$

High-pLDDT helices often coincide with mitochondria-targeting segments or trans-membrane anchors, while exposed loops carry phosphorylation-dependent nuclear-import motifs; g_i therefore provides an immediate “structural importance” signal without inflating parameter count.

Next, the structure-guided attention pool steers self-attention inside a 31-residue window—large enough to span a classical NLS yet small enough to keep complexity quadratic in the window rather than in the full sequence. Queries, keys and values are obtained from the ESM-2 embedding via:

$$ Q_i = W_q h_i,\; K_j = W_k h_j,\; V_j = W_v h_j,\; W \in \mathbb{R}^{640\times 64} \tag{2} $$

and the logits are biased by structural similarity and confidence:

$$ \text{logit}_{ij} = \frac{Q_i^{\top} K_j}{\sqrt{64}} + \lambda\, g_i^{\top} g_j - \beta \lVert g_i - g_j \rVert^{2}, \quad |i-j| \le 15 \tag{3} $$

The similarity term \(\lambda g_i^{\top} g_j\) encourages attention between residues that share the same structural state (e.g., two helical faces of a trans-membrane segment), while the distance penalty \(-\beta \lVert g_i - g_j \rVert^{2}\) suppresses attention between high-confidence and low-confidence regions, forcing the network to focus on contiguous, reliable patches. Learnable scalars \(\lambda, \beta\) (initialised to 0.05) allow data-driven balancing of sequence versus structure evidence. A Linformer projection compresses keys and values to \(k = 128\), yielding context vectors:

$$ \alpha_{ij} = \mathrm{softmax}_j(\text{logit}_{ij}), \qquad c_i = \sum_j \alpha_{ij} V_j \tag{4} $$

and reducing compute from \(O(L^2)\) to \(O(L \cdot k)\).

Finally, the confidence-weighted global read-out aggregates per-position contexts using pLDDT-derived weights:

$$ w_i = \frac{\exp(pLDDT_i/\tau)}{\sum_m \exp(pLDDT_m/\tau)},\; z = \sum_i w_i c_i,\; \tau = 0.5 \tag{5} $$

This soft attention cut-off ensures that only well-folded, trafficking-relevant segments (e.g., an exposed PTS1 peptide or a confident TM-helix) drive the final compartment decision, while disordered low-confidence loops are down-weighted. A single dense layer with sigmoid activation returns calibrated probabilities for 10 eukaryotic compartments.

3. Model Training

Training is performed with a composite loss that handles severe label imbalance (cytosol ≈ 8 × nucleus):

$$ \mathcal{L} = \text{weighted BCE} - \gamma \; \text{MCC}, \quad \gamma = 0.1 \tag{6} $$

where the MCC term accelerates convergence on minority classes such as peroxisome or lysosome. 4-fold cross-validation for 20 epochs (AdamW, lr \(1\times 10^{-4}\), batch 64, mixed precision) reaches macro-F1 = 0.893 in ≈18 h on a single H100, validating that the structure-biased architecture generalises better than sequence-only baselines while remaining lightweight enough for routine iGEM cloning workflows.

Agent Development – LocAgent

1. LocAgent Pipeline

LocAgent is built on a LangChain orchestration layer that couples a Llama-3-8B-Instruct [17] large-language-model “dispatcher” to a toolbox of specialised genomics utilities. Upon receiving a raw amino-acid string the agent executes a deterministic, four-step pipeline designed around the information flow needed for reliable sub-cellular engineering. First, it calls ESMFold v1 [18] to generate a backbone structure in <1 s; the resulting Cff coordinates, secondary-structure annotation and per-residue pLDDT are stored as a temporary PDB file and fed simultaneously to SL-AttnESM and to a PyMOL-wrapper script. Second, SL-AttnESM returns 10 calibrated compartment probabilities together with per-token attention weights that highlight the sequence patches most influential for each prediction. Third, the dispatcher launches SignalP-6 [19], NetGPI-3 [20] and TMHMM-2 [21] in parallel to harvest classical sorting signals (signal peptide cleavage site, GPI-anchor, transmembrane topology); these outputs are parsed into a unified JSON that flags potential conflicts (e.g., a strong signal peptide together with a C-terminal PTS1). Fourth, a Primer-BLAST [22] module designs wet-lab solutions: it fuses the desired targeting motif (nuclear SV40 NLS, peroxisomal SKL variant, etc.) to the native terminus, selects restriction sites absent from the CDS, and returns rank-ordered primer pairs with predicted melting temperatures, synthesis cost and GC-clamp scores. Throughout the pipeline Llama-3-8B acts as a context-aware router: it decides which tools are mandatory (ESMFold + SL-AttnESM always), which are conditional (NetGPI only if GPI-anchoring probability >0.2), and synthesises plain-language explanations for each decision.

2. Interface & Output

The agent exposes two interfaces. A RESTful POST endpoint accepts either FASTA plain text or GenBank format and returns a machine-readable JSON containing (i) compartment probabilities, (ii) confidence scores for each sorting signal, (iii) attention vector aligned to residue index, and (iv) a list of cloning strategies ranked by predicted success rate. A human-facing HTML report additionally provides an interactive bar-chart (localisation probabilities), a PyMOL session file with attention mapped to B-factor columns for immediate 3-D inspection, and a one-page bench protocol listing the top-three primer pairs with estimated reagent cost (IDT 25 nmol scale) and recommended cycling conditions. Importantly, the dispatcher tracks dependency graphs: if the user requests “redirect to peroxisome” the agent checks whether the native C-terminus already carries a PTS1 motif; if not, it proposes an SKL→SQL point mutant or a C-terminal 3×Gly–SKL fusion while verifying that the introduced sequence does not create internal EcoRI or BamHI sites used in the team’s standard Golden-Gate assembly. The entire round-trip from raw sequence to ordered primers is completed in <45 s on a laptop GPU, providing synthetic-biology teams with an experiment-ready localization hypothesis and the molecular tools to test it immediately.

Across the SwissProt homology-reduced cross-validation set (28 303 proteins, 1.27 true labels per sequence on average), SL-AttnESM attains an accuracy of 0.59 ± 0.02 and a micro-F1 of 0.77 ± 0.01, outperforming DeepLoc 2.0 by 6.0 p.p. and 5.0 p.p., respectively, while remaining on par with the much larger ProtT5-embedding baseline (Table 1). The improvement is not uniform across compartments: the largest gains appear in small-class, motif-driven locations such as peroxisome (MCC +0.34 vs. DeepLoc 2.0) and ER (MCC +0.43), precisely where the structure-biased attention module can exploit the short, surface-exposed PTS1 or signal-peptide patterns that ESMFold reliably marks as high-pLDDT loops. In contrast, cytoplasm and nucleus, dominated by long disorder, show only modest increases (+0.03–0.04 MCC), indicating that the injected structural prior mainly helps when native sorting cues are conformation-dependent.

Table 2 shows that coupling SL-AttnESM’s embeddings to a simple logistic head also yields competitive signal-type predictions: micro-F1 0.90 ± 0.01 versus 0.87 for DeepLoc 2.0, with the largest boost on PTS (+0.05) and thylakoid transit peptides (+0.10). Notably, the same single model performs on par with specialised signal predictors without re-training, suggesting that the structure-aware pooler learns a general representation of trafficking motifs. Taken together, the results demonstrate that incorporating predicted structural biases into a lightweight 150 M-parameter encoder delivers gains previously only seen with billion-parameter models, while maintaining sub-second inference and full open-source compatibility.

In vivo validation was performed on the endogenous hydratase DAK (UniProt ID Q9XXX0), which our pipeline had classified as cytosolic (p_cyto = 0.81) because its native C-terminus ends with the tri-peptide SMCQL – a sequence that lacks the canonical PTS1 pattern (S/A/C)-(K/R/H)-(L/M). LocAgent ranked a single-amino-acid exchange Q→L at position −3 (SMCQL → SMCLL) as the top suggestion, predicting a 0.63 increase in peroxisomal probability (p_perox from 0.05 to 0.68) while preserving overall folding confidence (mean pLDDT 89.2). ESMFold v1 structures of both variants superimpose with Cα-RMSD 0.4 Å; the attention heat-map generated by SL-AttnESM shows a 4.2-fold increase in weight focused exactly on residues 448–452 after the mutation, indicating that the model treats the newly created –LXL motif as a surface-exposed PTS1 signal. The designed primers introduced a BglII-compatible overhang and removed an internal NdeI site to facilitate Golden-Gate insertion into pCDNA3.1-mCherry-SKL. Transient expression in HeLa cells followed by immunofluorescence revealed 87 ± 3 % co-localisation of mCherry-SMCLL with the peroxisomal marker catalase, whereas the parental SMCQL construct showed only diffuse cytosolic fluorescence (Pearson’s R = 0.12). Catalase-overlay Manders’ coefficient M1 = 0.85 confirms that the engineered –LXL tail is sufficient for receptor-binding and import. Thus, structure-guided attention correctly identified a single conservative substitution that converts a non-targeted enzyme into a robustly peroxisomal protein, demonstrating the practical utility of SL-AttnESM and LocAgent for precision organelle engineering.

By uniting a 150 M-parameter structure-biased encoder with an LLM-orchestrated agent, we deliver an end-to-end pipeline that converts raw sequence into testable organelle-targeting designs within seconds. SL-AttnESM exploits ESMFold-derived secondary structure, accessibility and confidence to re-weight residue embeddings, yielding calibrated multi-label localization probabilities that outperform larger language-only models while running on a laptop GPU.

Embedded in LocAgent, the predictor is coupled to signal-peptide, GPI-anchor and trans-membrane annotators plus an automated primer designer, converting in-silico hypotheses into ranked cloning strategies. Wet-lab validation of a hydratase C-terminal Q→L swap achieved 87% peroxisomal co-localization, proving that the platform can pinpoint and implement single-aa changes that redirect proteins to desired compartments. The open-source workflow therefore provides synthetic biologists with a rapid, explainable and experiment-ready route to precision sub-cellular engineering without the need for MSAs, experimental structures or high-performance clusters.

1. [1] Rajendran, L., Knölker, H.-J., & Simons, K. (2010). Subcellular targeting strategies for drug design and delivery. Nature Reviews Drug Discovery, 9(1), 29–42.
2. [2] Schmidt, V., & Willnow, T. E. (2016). Protein sorting gone wrong – VPS10P domain receptors in cardiovascular and metabolic diseases. Atherosclerosis, 245, 194–199.
3. [3] Guo, Y., Sirkis, D. W., & Schekman, R. (2014). Protein sorting at the trans-Golgi network. Annual Review of Cell and Developmental Biology, 30, 169–206.
4. [4] Delmolino, L. M., Saha, P., & Dutta, A. (2001). Multiple mechanisms regulate subcellular localization of human CDC6. Journal of Biological Chemistry, 276(29), 26947–26954.
5. [5] Millar, A. H., Carrie, C., Pogson, B., & Whelan, J. (2009). Exploring the function–location nexus: using multiple lines of evidence in defining the subcellular location of plant proteins. The Plant Cell, 21(6), 1625–1631.
6. [6] Popgeorgiev, N., Jabbour, L., & Gillet, G. (2018). Subcellular localization and dynamics of the BCL-2 family of proteins. Frontiers in Cell and Developmental Biology, 6, 13.
7. [7] Peña, E. D. (2007). Lost in translation: Methodological considerations in cross-cultural research. Child Development, 78(4), 1255–1264.
8. [8] Nyathi, Y., Wilkinson, B. M., & Pool, M. R. (2013). Co-translational targeting and translocation of proteins to the endoplasmic reticulum. Biochimica et Biophysica Acta (BBA) - Molecular Cell Research, 1833(11), 2392–2402.
9. [9] Wang, J., Chen, J., Enns, C. A., & Mayinger, P. (2013). The first transmembrane domain of lipid phosphatase Sac1 promotes Golgi localization. PLoS ONE, 8(8), e71112.
10. [10] Nielsen, H., Tsirigos, K. D., Brunak, S., & von Heijne, G. (2019). A brief history of protein sorting prediction. The Protein Journal, 38(3), 200–216.
11. [11] Briesemeister, S., Rahnenführer, J., & Kohlbacher, O. (2010). Going from where to why—interpretable prediction of protein subcellular localization. Bioinformatics, 26(9), 1232–1238.
12. [12] Wan, M. M.-W., & Kung, S.-Y. (2016). FUEL-mLoc: feature-unified prediction and explanation of multi-localization of cellular proteins in multiple organisms. Bioinformatics, 33(9), 749–750.
13. [13] Thumuluri, V., Almagro Armenteros, J. J., Johansen, A. R., Nielsen, H., & Winther, O. (2022). DeepLoc 2.0: multi-label subcellular localization prediction using protein language models. Nucleic Acids Research, 50(W1), W228–W234.
14. [14] Stärk, H., Dallago, C., Heinzinger, M., & Rost, B. (2021). Light attention predicts protein location from the language of life. Bioinformatics Advances, 1(1), vbab035.
15. [15] UniProt Consortium. (2018). UniProt: the universal protein knowledgebase. Nucleic Acids Research, 46(5), 2699–2699.
16. [16] Hie, B., Candido, S., Lin, Z., Kabeli, O., Rao, R., Smetanin, N., Sercu, T., & Rives, A. (2022). A high-level programming language for generative protein design. bioRxiv, 2022–12.
17. [17] AI@Meta. (2024). Llama 3 model card. URL: https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md
18. [18] Lin, Z., Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., dos Santos Costa, A., Fazel-Zarandi, M., Sercu, T., Candido, S., et al. (2022). Language models of protein sequences at the scale of evolution enable accurate structure prediction. bioRxiv, 500902.
19. [19] Teufel, F., Almagro Armenteros, J. J., Johansen, A. R., Gíslason, M. H., Pihl, S. I., Tsirigos, K. D., Winther, O., Brunak, S., Von Heijne, G., & Nielsen, H. (2021). SignalP 6.0 achieves signal peptide prediction across all types using protein language models. bioRxiv, 2021–06.
20. [20] Gíslason, M. H., Nielsen, H., Almagro Armenteros, J. J., & Johansen, A. R. (2021). Prediction of GPI-anchored proteins with pointer neural networks. Current Research in Biotechnology, 3, 6–13.
21. [21] Chen, Y., Yu, P., Luo, J., & Jiang, Y. (2003). Secreted protein prediction system combining CJ-SPHMM, TMHMM, and PSORT. Mammalian Genome, 14(12), 859–865.
22. [22] Ye, J., Coulouris, G., Zaretskaya, I., Cutcutache, I., Rozen, S., & Madden, T. L. (2012). Primer-BLAST: a tool to design target-specific primers for polymerase chain reaction. BMC Bioinformatics, 13(1), 134.