LLMs on Ray and LanceDB - Scaling Multimodal Pipelines and Native Ray APIs for LLM Inference

This meetup recording features experts from LanceDB and Anyscale discussing advanced techniques for scaling LLM workloads. Learn how to handle multimodal data challenges as Jonathan Hsieh demonstrates how LanceDB's architecture solves multimodal data storage problems with features like zero-cost schema evolution that enhance AI engineer productivity. Discover how Ray and Ray Data can help scale and streamline data processing pipelines. Richard Liaw then explores Native Ray APIs for LLM inference, addressing the complexities of working with open-source LLMs and introducing two powerful new APIs from the latest Ray release: Batch LLM Inference with Ray Data LLM and Online LLM Inference with Ray Serve LLM. Gain insights from this collaboration between Anyscale and LanceDB designed to improve the AI development experience.