Explore the evolving landscape of AI coding agents in this 35-minute conference talk that addresses the polarized debate surrounding their effectiveness in serious programming. Examine the three distinct eras of AI coding tools, from GPT-3's text completion capabilities in 2022 to ChatGPT's ragbot implementations in 2023, culminating in today's autonomous agent era that enables direct file editing, command execution, and external service interaction. Learn why the most significant mistake developers make is applying outdated mental models to new agent capabilities, and discover how the rapid advancement in model capabilities requires completely new strategies for effective utilization. Watch a live demonstration of Sourcegraph's Amp agent showcasing autonomous edits, tool use, sub-agents, planning and execution, and nuanced problem-solving capabilities. Master best practices from power users including crafting detailed prompts, establishing effective feedback loops, leveraging code understanding features, and conducting thorough code reviews. Identify common anti-patterns such as micromanagement and under-prompting while understanding how to implement parallel agents effectively. Gain insights into the controversial design philosophy behind modern coding agents, including their autonomous editing capabilities and adherence to Unix philosophy principles. Understand why coding agent proficiency represents a high-ceiling skill that requires continuous adaptation to rapidly evolving capabilities, and learn empirical strategies for employing agents to complete complex tasks in production codebases.

Syllabus

[00:14] Current Discourse: The talk begins by acknowledging the polarized debate on AI's role in coding. While some elite programmers are skeptical, many developers find significant value in AI tools, suggesting a disconnect between top-tier and mainstream experience. Liu frames the discussion by referencing opinions from figures like Jonathan Blow and Eric S. Raymond, highlighting the varied perspectives in the field.
[03:01] Paradigm Shift: The most significant mistake developers make is using new agents with old mental models. Liu emphasizes that we are in a "step function transition" in model capabilities, meaning that strategies from even six months ago are already outdated for leveraging the full power of today's agents.
[05:06] GPT-3 Era 2022: This era was defined by text completion models. The primary application was "copilot" or "autocomplete," where the AI would suggest the next few lines of code based on the preceding context.
[05:24] ChatGPT Era 2023: The introduction of instruct-tuned models like GPT-3.5 led to the rise of chatbots. In the coding world, this manifested as "ragbots," which combined a chat interface with a retrieval engine to answer questions about a codebase.
[06:11] Agent Era Present: The current era is defined by models capable of tool use and autonomous operation. This requires a new application architecture where the agent can directly edit files, run commands, and interact with external services to accomplish a goal.
[07:27] Autonomous Edits
[09:55] Unix Philosophy
[10:24] New Applications
[13:15] The Task
[14:30] Tool Use
[15:53] Sub-Agents
[17:56] Planning & Execution
[19:46] Nuanced Problem Solving
[23:21] Detailed Prompts
[24:21] Feedback Loops
[28:03] Code Understanding
[28:36] Code Reviews
[30:35] Micromanagement
[30:46] Under-prompting
[31:52] Parallel Agents
[33:18] High-Ceiling Skill