Build Computing Olympiad Agents with LangGraph

In this tutorial, we create Olympiad programming agents using LangGraph, drawing upon the techniques and benchmark dataset introduced in the paper "Can Language Models Solve Olympiad Programming?" by Quan Shi, Michael Tang, Karthik Narasimhan, and Shunyu Yao. #AI #LangGraph #llm Throughout the tutorial, we learn how to enhance the agent's performance by incorporating three key techniques: 1. Reflection: In the first part, we develop a zero-shot tool calling agent and prompt it to reflect on the test case results, enabling it to correct its initial errors. This agent is comparable to the one reported in the paper, achieving a pass rate of 12.38 on the USACO benchmark. 2. Retrieval: In the second part, we introduce an initial retrieval step, functioning as an "episodic memory" for the agent. This step retrieves high-quality few-shot examples from our corpora of programming problems, assisting in solving the bronze level question. The agent created in this part is similar to the one benchmarked at 20.2 in the paper. 3. Human-in-the-loop: In the third part, we utilize the `interrupt_after` feature to allow the user to copilot the agent, guiding it towards a better answer. The benchmark performance in this scenario is limited only by the competitiveness of the human collaborator. By the end of this tutorial, you will have a solid understanding of how to build agents in LangGraph, leveraging advanced techniques such as reflection, retrieval, and human-in-the-loop interaction. Chapters: 00:00 Introduction 01:30 Overview of the USACO Dataset and Benchmark Performance 02:38 Introduce the 3 techniques 06:01 Part 1: Zero-Shot Agent with Reflection Capabilities 12:27 Review Zero-Shot Results 13:33 Part 2: Add Episodic Memory Retrieval 18:48 Review Results of Augmented Agent 21:06 Part 3: Add Human-in-the-Loop 26:43 Review Copilot Results 27:44 Conclusion Additional Resources: - Research Paper: https://arxiv.org/abs/2404.10952v1 - Tutorial Code: https://langchain-ai.github.io/langgraph/tutorials/usaco/usaco/ - Reflection Agents: https://www.youtube.com/watch?v=v5ymBTXNqtk Presenter: https://twitter.com/WHinthorn