Multimodal Generative AI: Technology Overview and Business Implications

Explore multimodal generative AI in this comprehensive 1-hour 38-minute conference talk from Microsoft Reactor Bengaluru. Delve into the technical aspects of training generative AI systems that handle multiple input types simultaneously, including text, image, and audio. Learn about business applications, limitations, and associated costs of these advanced systems. Gain insights into the open-source LLaVA (Large Language-and-Vision Assistant) multimodal system. Discover key concepts such as data gathering, outliers, nonlinearities, and the differences between statistics and AI. Examine practical examples like analog gauges and conversational systems. Understand the architecture, training data sets, and challenges like catastrophic forgetting and repeatability crisis. Investigate advanced topics including eigenvalue decomposition and visual inspection techniques. Access the accompanying presentation slides for a deeper understanding of the material covered.