LLMs as Unified Multimodal Learners for Clinical Prediction: Converting All Patient Data into Text

A new study published on arXiv (2607.15380) proposes a simpler alternative to traditional clinical prediction systems: convert all patient data, regardless of modality, into a single natural language sequence and fine-tune a pretrained large language model (LLM).

Electronic health records combine free-text clinical narratives with structured measurements such as vital signs, laboratory values, and comorbidities. Yet most clinical prediction systems still rely on task-specific fusion architectures, pairing dedicated encoders for each modality with learned combination mechanisms that must be re-engineered for every new task and clinical setting.

The proposed approach eliminates this complexity by treating all patient data as text. This could make medical AI systems more adaptable and easier to deploy across different hospitals and conditions, without requiring separate tools for each type of data.

If you're curious about the technical details, you can read the full study on arXiv. The key takeaway is that this method could lead to more flexible and powerful clinical prediction models, potentially reducing development costs and improving accuracy.