Project 4: AI Coding Agent

AI coding agents are, as the meme above puts it, suddenly everywhere. In Project 3 (docchat) you built the start of one: a program that loads your files into a conversation and answers questions about them. Every tool it has so far only reads your computer: cat dumps a file, grep searches for text, ls lists a directory. The model can read your code and tell you what is broken, but it cannot change a single character.

This project adds the tools that write: creating files, editing them, deleting them, and committing each change to git. Once docchat can do that, it stops being a program you chat with and becomes an autonomous agent that edits your project for you. It is the same jump that separates a chatbot from a coding agent like Claude Code or Cursor, and you are building a small version of exactly that.

Due: Wednesday, December 2, see the schedule.

Learning objectives

1. extend smaller projects into larger projects
2. understand how AI agents work

Instructions

1. Your project must meet every specification from Project 3 (docchat). If you missed any points there, fix the issue now, or you will lose those points again on this project.

2. Create a new branch in your repository and do all of your work in it. You may not touch the main/master branch you submitted for docchat.

3. Coding tasks.

   When your program starts, it should run two checks:

   • Look for a .git folder in the current directory. If it is not present, print an error and stop.

   • Look for an AGENTS.md file in the current directory. If it is present, load it into the conversation using your cat tool.

     Note: AGENTS.md is a standard file that AI agents load when they start working in a repo. Think of it as a README written for the agent instead of for a person: it lists project-specific instructions the agent should follow. You can find details and examples at https://agents.md/.

   Then add the following four tools.

   doctests

   • It takes a single argument, path.

   • It runs the doctests in path with the --verbose flag and returns the output.

     Note: Explicit confirmation that the test cases are passing is helpful for the LLM, which is why we ask for --verbose output here rather than a silent success.

     Note: We have not covered how to run doctests from inside a Python program. There are several ways to do it, and any one you choose is fine.

   write_file

   • It takes three arguments: path, contents, and commit_message.

   • It opens path and writes contents to it. The output file must be UTF-8 encoded.

   • Then it uses the git library in Python to run the equivalent of:

     $ git add <path>
     $ git commit -m "[docchat] <commit_message>"

     Note: It is standard practice to have an agent commit every change it makes to disk with git. That way, when your agent “goes Skynet” on you, it is easy to travel back in time and undo its work. (AI researchers really do reach for Terminator memes when they talk about this.)

     

   • If the file is a Python file, run your doctests tool on it and return the output.

   write_files

   • It takes two arguments: files and commit_message.

   • files is a list of dictionaries, each with a path key and a contents key. Write each one the same way write_file does.

   • After writing all of the files, add and commit them together.

     Note: We recommend writing the “real” code once, in write_files, and making write_file a thin wrapper that calls it with a one-element list. Less code is better than more code: it is easier to write, easier to test, and easier to change later. This is the DRY principle, short for Don’t Repeat Yourself.

     

     Note: So why keep write_file at all if it is just a wrapper? Because LLMs, like people, use a specific tool more reliably than a general one. Just having the specific write_file in the toolbox helps the model work out when and how to reach for the general write_files.

   rm

   • It takes a single argument, path.

   • It deletes the path using Python’s os.remove function.

   • It supports multiple files at once using globs.

   • It creates a commit with the file removed, using the commit message [docchat] rm <path>.

     Warning: rm is dangerous when it is implemented incorrectly. Two cautionary tales:

     • Meta’s director of AI alignment research, Summer Yue, had Claude accidentally delete her inbox through a bad combination of the rm and compact commands: https://www.pcmag.com/news/meta-security-researchers-openclaw-ai-agent-accidentally-deleted-her-emails.
     • Toy Story 2 was almost lost forever to a bad rm command combined with a glob: https://thenextweb.com/news/how-pixars-toy-story-2-was-deleted-twice-once-by-technology-and-again-for-its-own-good.

   Warning: Every one of these tools must validate its path before running, exactly like your read tools did: it must not run on absolute paths or on paths containing ... When your read tools had this bug, the worst case was leaking a file to the model. These tools have write permission, so the worst case is now a rogue agent deleting everything on your computer. Get the check right.

4. Repository organization.

   Add examples to your README that show the agent in action: creating, modifying, and deleting files, and making git commits along the way. Use shell commands like ls, cat, and git to show the before and after, and format the prompts clearly.

   One possible example is the session below, which shows that docchat can create a file when asked, and that the new file is added to the git repo automatically:

   $ git checkout -b agent-work
   Switched to a new branch 'agent-work'
   $ ls -a
   .git  AGENTS.md  README.md
   $ git log --oneline
   c21103f (HEAD -> agent-work, master) init commit
   $ docchat
   chat> Create python code that implements the project in README.md
   Created the file hello_world.py
   chat> ^C
   $ ls -a
   .git  AGENTS.md  README.md  hello_world.py
   $ git log --oneline
   3cfb0a6 (HEAD -> agent-work) [docchat] create basic hello world python project
   c21103f (master) init commit

Grading rubric

This project is worth 16 points.

• Your project meets every specification from Project 3 (docchat).
• All of your work is done in a new branch, leaving the main/master branch you submitted for docchat untouched.
• Coding tasks: your program runs the two startup checks and adds the doctests, write_file, write_files, and rm tools, each of which validates its path before running.
• Repository organization: your README shows the agent creating, modifying, and deleting files, and making git commits along the way.

You may still complete any of the extra credit from previous projects, though extra credit you have already earned will not be counted twice. There are also new extra credits below, specific to this project.

Extra credit

• (1 pt) A pip_install tool. It takes a single parameter, library_name, and runs the pip3 install <library_name> command.

  Warning: Recall that a PyPI package can contain arbitrary code. Any agent that can install packages can therefore choose, on purpose or by accident, to destroy your computer. For example, the popular LiteLLM package suffered a recent supply-chain attack:

  • https://docs.litellm.ai/blog/security-hardening-april-2026
  • https://www.trendmicro.com/en_us/research/26/c/inside-litellm-supply-chain-compromise.html
  • https://console.groq.com/docs/litellm

  Open question: how would you prove that a piece of code is “safe”?

• (2 pts) The Ralph Wiggum loop. Whenever your doctests fail, force the agent to do another round of tool use, and only leave the loop once the doctests pass. You may leave the loop always on, or make it something you can turn on and off.

  Note: Anthropic recently renamed their Ralph Wiggum implementation to just “Ralph” for copyright reasons. You can follow along in their GitHub repo: https://github.com/anthropics/claude-plugins-official/commit/44328beed48874d8e00da6c4ca5daaa5f0f3183c.

• (4 pts) Have your agent complete the markdown compiler on its own. To claim this, include in your README a link to a branch of your markdown compiler project that contains only commits made by your AI agent. The easiest approach is to start a brand-new repo rather than reuse your existing one.

  Hint: The Llama models on the free Groq API are not great at this task: the Llama models are old, and the free Groq tier has severe token limits. It is much easier with a state-of-the-art (SOTA) model like OpenAI’s latest GPT or Anthropic’s latest Opus. An OpenRouter API key lets you experiment across models. Finishing this task with the newest, most expensive model should cost less than $0.10, but put about $10 of credit on the account so you have room to play around.

• (4 pts) Let your agent update files instead of only overwriting them. Change write_file / write_files so that the contents parameter becomes optional and a diff parameter can be given instead. (A diff is the standard format for describing the changes to a file; applying one to produce the updated file is called patching the file.)

  Even SOTA LLMs are bad at generating correct diffs, mostly because a diff needs line numbers and LLMs are bad at counting which line a piece of code is on. (We can more or less prove that counting will always be hard for them.) The practical consequence is that you cannot use the standard diff/patch tools to apply the model’s output. There are many workarounds, and you are welcome to use whatever you can get to work. We recommend wiggle, a tool built for applying exactly these “broken” LLM-generated diffs.

  Why bother with updates instead of rewrites? For a large file, rewriting the whole thing on every edit wastes a lot of tokens, which makes the agent more expensive to run. Worse, LLMs often make small typos, like dropping a ) that should match a (, and those typos make the code look correct while behaving very wrongly.

Submission

Submit a link to the new branch of your GitHub repo on Gradescope.

Along with the link, submit a one-to-two-sentence explanation of what you believe your grade should be. In particular:

• If you completed any extra credit, say so.
• If any part of your project does not work, say so, I may grade more leniently when you are upfront about it.
• If you are requesting points back from a previous project, you must say so explicitly.