AI, Data Science & Cloud Certificates from Google, IBM & Meta — 40% Off
One plan covers every Professional Certificate on Coursera. 40% off your first 3 months — limited time.
Unlock All Certificates
Explore a groundbreaking conference talk that presents a novel multimodal framework leveraging Large Language Models for disease prediction by integrating structured Electronic Health Records and wearable time series data. Learn how this innovative system addresses unique healthcare challenges including temporal dynamics, heterogeneous data formats, and clinical interpretability requirements through modality-specific encoders that transform input streams into compact latent representations within a unified embedding space. Discover the end-to-end training approach that enables the LLM to develop rich, context-aware representations linking current behavioral signals to broader clinical trajectories, while supporting auxiliary context such as demographics and prompt instructions for dynamic adaptation to specific tasks or patient profiles. Examine the evaluation results from UK Biobank data involving approximately 70,000 participants, demonstrating superior performance over single-modality baselines with meaningful wearable data influence on predictions when integrated with EHR data (correlation r = 0.771). Understand the modular architecture's extensibility potential for incorporating additional data sources like nutrition or imaging through corresponding encoders, and gain insights into how LLMs can evolve into adaptive, multimodal engines for real-time, patient-centric care supporting earlier interventions, continuous monitoring, and personalized clinical decision-making.
