How To Trust a Liar - Instrumenting AI Execution With OpenTelemetry

Learn how to build observable AI systems by exploring the development of Commit Story, an open source automated engineering journaling system that demonstrates practical AI instrumentation techniques. Discover how to implement OpenTelemetry to monitor AI behavior in real-time, capturing correlated traces, logs, and metrics as an AI agent processes git commits and generates engineering journal entries. Examine a real-world debugging scenario where development-time telemetry helped identify performance issues, ruled out incorrect theories, and guided an AI coding assistant to avoid duplicate logic while verifying fixes. Explore the unique approach of enabling AI agents to query their own telemetry data during development, using runtime evidence to make better coding decisions. Gain insights into the tradeoffs involved in AI observability and access hands-on demonstrations with downloadable software for practical experimentation.