Explore enterprise-level retrieval solutions in this 52-minute conference talk from AICamp London that addresses the complexities of scaling AI applications, particularly RAG chatbots. Learn how to overcome common pitfalls when dealing with multi-modal data, ambiguous user intent, and long reasoning chains through Desperia's innovative tensor database technology. Discover practical approaches to storing, querying, and leveraging institutional knowledge for precise and efficient AI systems. The session covers the four key problems in industrial AI deployment, handling unstructured data with embeddings and context knowledge, and addresses critical challenges including multi-modality, precision requirements, enterprise system integration, and cost management. Watch live demonstrations featuring complex PDF and medical document retrieval systems, including interactive Q&A sessions on drug mechanisms and comparisons. Gain insights into semantic search fundamentals using dense vectors, advanced retrieval techniques with late interaction and sparse vectors, and strategies for integrating data preferences through sparse tensors and re-ranking. Understand lexical search and ranking algorithms, learn to build comprehensive retrieval implementations combining multiple methods with tensor mathematics, and explore rank profiles for training precision-focused re-rankers. The presentation includes evaluation methodologies for retrieval systems, the role of knowledge graphs in contextual awareness, and approaches to context-aware knowledge bases with multi-modal retrieval capabilities. Examine normalization and ranking strategies, vector database implementation, AI pipeline construction, and future product directions in enterprise AI deployment.

Syllabus

0:00 - Introduction to Desperia and Enterprise Retrieval
0:49 - Challenges in Building and Scaling AI Applications
1:56 - The Four Key Problems in Industrial AI Deployment
3:41 - Handling Unstructured Data with Embeddings and Context Knowledge
4:48 - Key Challenges in Enterprise AI: Multi-modality, Precision, Enterprise Systems, and Cost
5:54 - Real-World Applications: Medical and E-commerce Retrieval
6:44 - Demo: Complex PDF and Medical Document Retrieval with Co-Valley
9:32 - Interactive Demo and Q&A: "Mechanism of Action" and Drug Comparison
12:10 - The Fundamentals of Semantic Search with Dense Vectors
14:21 - Advanced Retrieval: Late Interaction and Sparse Vectors
20:16 - Integrating Data and Preferences: Sparse Tensors and Re-ranking
24:33 - Lexical Search and Ranking Algorithms
26:58 - Building a Retrieval Implementation: Combining Methods and Tensor Math
28:44 - Rank Profiles and Training Re-rankers for Precision
29:49 - Evaluating Retrieval Systems: Metrics, Judgment Lists, and Customization
34:13 - The Role of Knowledge Graphs and Contextual Awareness
39:44 - Context-Aware Knowledge Bases and Multi-Modal Retrieval
44:08 - Normalization, Ranking, and Evaluation Strategies
46:40 - Vector Databases and Building an AI Pipeline
48:23 - Product Offerings and Future Directions