Coursera Flash Sale
40% Off Coursera Plus for 3 Months!
Grab it
Explore the critical infrastructure components that power generative AI at the edge in this 24-minute conference talk. Learn why focusing on backbone toolchains rather than just models leads to better performance, lower costs, and reduced latency in AI deployments. Examine a real-world cloud-grade, on-premises AI appliance built on Qualcomm AI 100 Ultra cards to understand what drives speed and affordability in edge AI systems.
Discover the platform layer fundamentals including Linux-based reliability, containerized deployment, observability, and security measures that create a solid foundation. Dive deep into the performance engine architecture featuring a compiler that maps LLM graphs onto 64 NPUs, advanced decoding techniques like speculative decoding and prefix caching, and runtime integrations with PyTorch, ONNX, and VLLM for continuous batching and multi-tenant serving.
Master developer-friendly tools including OpenAI-compatible APIs for LLMs, VLMs, embeddings, and indexing, plus visual development environments like Langflow for building RAG pipelines without complex integration code. Compare pipeline parallelism, tensor parallelism, and hybrid strategies to understand when each approach delivers optimal results.
Address the common challenge of fine-tuning without power-hungry GPU farms through parameter-efficient methods that enable a 150-watt card to fine-tune models up to approximately one billion parameters, making private customization accessible for SMBs and small teams. Gain practical insights for implementing private, low-latency GenAI solutions for safety-critical applications, robotics, and enterprise knowledge systems.
