What question did this study set out to answer?

To evaluate the effectiveness of vision-language models in defining structural segments from MSA images for the PDB-Descriptome project.

March 28, 2026Open Access

VLMs struggle to extract definitions of structural segments from MSA images: toward the design of a human-in-the-loop annotation pipeline for the PDB-Descriptome project

Key Points

To evaluate the effectiveness of vision-language models in defining structural segments from MSA images for the PDB-Descriptome project.
Used synthetic MSAs with annotated structural biological entities and referring expressions.
Evaluated two vision-language models under naïve and strict prompts.
Compared VLMs' performance against human annotator results.
VLMs excelled in extracting referring expressions but struggled with defining SBE boundaries.
Human annotators achieved higher boundary precision with slightly lower textual accuracy.
Supports the need for a human-in-the-loop annotation pipeline for improved reliability.

Abstract

Abstract We investigate the ability of vision–language models (VLMs) to extract structural region definitions from rasterized multiple sequence alignment (MSA) images, as part of the PDB-Descriptome project. Using synthetic MSAs with annotated structural biological entities (SBEs) and structural biological Referring Expressions (SBREs), we evaluate two VLMs, gemini-2.5-flash and gemini-2.5-pro, under naïve and strict prompts. While VLMs perform well in SBRE extraction, they show poor accuracy in defining SBE boundaries. In contrast, a human annotator achieves high boundary precision with slightly lower textual accuracy. These results support a human-in-the-loop pipeline for reliable structure–text annotation.

Perguntar à IA

Bookmark

View Full Paper