Amp vs Claude Code for Infra

A conversation about the results with Sourcegraph CEO Quinn Slack

After a successful OSS task with Amp Code, I interviewed Sourcegraph CEO Quinn Slack. Now, I'm testing Amp against Claude on a new infrastructure task.

Claude Code vs Amp for Infra

Why Infra First?

I started with an OSS task, not infrastructure. I tested infra next because Bryan Bischof called Amp horrible at it.

Bryan is a strong developer, so I took his opinion seriously. His experience differed from mine, and I wanted to know why. I also hate infra, where an agent's help is most valuable.

The task was simple, not a production deployment. It was designed to reveal key differences.

The Task And Prompt

The task had four steps:

1. Create a simple, functional FastAPI todo app with HTMX.
2. Dockerize the app.
3. Deploy to Fly.io and help me decide between it and Railway.
4. Set up CI with GitHub Actions.

The exact prompts are at the bottom.

The Comparison Table

Legend:

• ✅ Liked
• ❌ Didn't like
• ⚠️ Could be improved

My Thoughts on the Results

The Good

Both models completed the task. I received working todo apps hosted on Fly.io from each.

The Bad

• Health Checks: Both used a deployment health check. Amp's worked. Claude's had a critical bug.

• The Database: I asked for a "simple" app but did not specify data storage.

  • Claude Code used in-memory storage, which works only for a toy example.
  • Amp performed better. It asked whether to use in-memory storage or SQLite for persistence. I chose SQLite and appreciated the question.
  • This choice mattered. Amp set up a persistent Fly volume. Claude, using in-memory data, did not.

• Taste: I preferred Amp on several minor points.

  • Amp put CSS in a separate file instead of embedding it in HTML.
  • Claude Code smartly checked if the app exists before deploying in CI.
  • Amp verified the health check in CI after deployment, another good step.
  • Amp chose a better memory setting, though the cost difference is negligible.

• Extra CI: Amp added CI steps for linting and testing, which is my preference. I want agents to automate testing. Other developers might disagree.

The Ugly

Both models failed a critical step: testing Docker locally before deployment. Claude Code didn't even try. Amp tried, failed, and deployed anyway without asking me. This was a major failure.

Prompts

Initial Prompt

Create a simple todo app that I can run locally using fastapi in htmx. It should be simple as it's an example app. Once done I will want to dockerize it, deploy to either railway or fly.io, an then create CI with github actions to automate the deployment. Make a plan for me to review that has all the neccesary steps and decisions, and present that plan to me to review before doing any work.

The models diverged from here. Amp asked about persistent storage, so I provided more details. Claude never asked.