ProtUrim: Autonomous Textual-Gradient Search for\\Protein Language Model Adaptation

Fine-tuning protein language models (PLMs) for fitness prediction requires choosing a structured recipe: low-rank adaptation (LoRA) rank, adapter placement, learning rate, data curriculum, and training budget. Standard practice often validates such choices with held-out masked language modeling (MLM) loss, but on BLAT\_ECOLX, a beta-lactamase assay, this proxy is inverted: MLM eval-loss is positively correlated with deep mutational scanning (DMS) Spearman~$\rho$ ($r{=}+0.77$), so minimizing perplexity selects poor downstream recipes. We introduce ProtUrim, an autonomous textual-gradient search procedure that uses DMS assay feedback to choose both the next fine-tuning recipe and the search move. Under a matched BLAT assay-feedback loop, Successive Halving (SHA) reaches $\rho{=}0.502$, while ProtUrim reaches $\rho{=}0.481\pm0.005$, 17\% above TextGrad recipe-only updates. Extending the search space to model choice, ProtUrim multi-model (ProtUrim-MM) reaches $\rho{=}0.610$ at ESM2-35M, exceeding SHA-35M ($\rho{=}0.568$) at lower GPU cost. Post-ESM2 panels with ESM3-open and ESMC show that model-family choice is assay-dependent, motivating joint search over assay, backbone, and adaptation recipe rather than reliance on a single default PLM.