Multimodal Learning on Graphs for Biological and Natural Systems

Watch a Harvard CMSA conference talk exploring advanced multimodal learning techniques for biological and natural systems through graph-based approaches. Delve into the challenges of modeling data with geometric relationships across different modalities, including biological sequences, chemical constraints, and 3D spatial interactions. Learn about structure-inducing pretraining frameworks that enhance relational structure in pretrained language models and discover novel graph pretraining objectives. Explore practical applications in genomic medicine and therapeutic science, including TxGNN for zero-shot prediction across 17,000 diseases and PINNACLE, a contextual graph AI model for 3D protein structure analysis and drug effect predictions at single-cell resolution. Understand the implementation of knowledge graphs, disease pooling, evaluation setups, and the development of tools for domain experts through integration, collaboration, and plain text interfaces.