Tech

Agentic Development: How to scale it?

October 25, 2026

My development these days goes like this: I talk with an AI to create a fairly large GitHub Issue, let the AI carry it through to PR creation and AI review, and finally review the parts that need a human myself. I feel like I have been using the ideas of loop engineering and graph engineering reasonably well in my own way.

But when I think about scaling this up to much larger software development, there is a lot I do not understand: how to configure the agents, what the unit of a task should be, and what role the human should play. So I read a few recently discussed examples of building software with AI to the fullest, and tried to sort things out. For each article, I will leave a summary and my own thoughts. Here is what this post covers.

Bun: An Expert Cuts the Phases, and Each Phase Runs as a Workflow

First, Rewriting Bun in Rust. It went viral this July. Bun moved 530,000 lines of Zig to Rust in 11 days. The diff was +1 million lines and about 6,800 commits. He designed and ran about 50 Dynamic Workflows (a Claude Code feature that creates and runs workflows whose steps can be carried out by agents), and at peak 64 Claudes ran in parallel. He used a pre-release Fable 5, and the cost was about $165,000 at API prices. The author estimates that doing it by hand would take 3 people about a year. He judged a gradual migration to be too costly, so he migrated all at once. As far as I can tell, he verified the migration using language-independent tests.

As I read it, the first step was converting 1,448 Zig files into the same number of Rust files. He spent 3 hours with Claude working out the migration rules (how to use Rust’s features), tried them on 3 files, and then migrated all 1,448 files. The unit of a task was one file, and it seems these ran in parallel. After that, the unit became each single compiler error, again fixed in parallel. The next step was to turn all the existing tests green locally, with each test file as a unit of work. Finally, he ran the tests prepared for each platform through CI and kept fixing until they passed.

In each phase and task, he ran two or more adversarial reviews (a review that thoroughly points out why the plan or code would not work) in separate contexts, and looped on the fixes. I do something similar in my own work, and it is close to my own best practice.

Migrating a language sounds like a clear goal, but it was not done by giving the AI a few lines of instruction. A person kept cutting the scope of the next phase step by step (probably through dialogue with the AI) and had the AI do it, and the pile of these phases got them to the end. I feel the same from my daily experience: the AI makes plans that look good, but they often miss things, so expert knowledge and experience still seem important here.

Also, each phase is not fully agentic: planning and execution are separated. In planning, he runs small experiments interactively to get a feel for what works, then defines a workflow with agentic steps using Dynamic Workflow (the Claude Code feature mentioned above), and runs it. The image is: make use of the agentic nature and the brain of the LLM, but keep the key points under control. My take is that this reduces variance, makes it easier to compare and improve, and also makes it easier for the agent itself to improve the workflow.

GitHub Copilot: The Same Migration, but Split So Production Keeps Running

Migrating the GitHub Copilot runtime to Rust, using Copilot. This also made big news, and it is from this September. It is TypeScript to Rust, but in contrast to Bun, it kept backward compatibility (replacing TypeScript parts with a thin layer that calls Rust, then deleting the old code) and rewrote things little by little in 128 PRs (fewer than I expected). The resulting Rust is about 830,000 lines, and mainly one developer did it over a few months (early May to late August?). The author says it would traditionally have taken a whole team one to two years.

They used many different models, and I can tell the models changed over time, but I cannot tell the details. They split the roles between a Primary Agent and Subagents, and Haiku and Gemini showed up often among the Subagents, so I guess they had to watch cost as well. For session.ts, the heaviest file, there is a detailed explanation: it was split into 15 phases, and each phase was migrated using 5.6 Sol or Opus 4.8.

As for how they cut tasks, judging from the time it took (a little under four months?) and how session.ts was migrated, the constraint of keeping backward compatibility and the existing development ecosystem seems large, unlike Bun. I understood it as an approach along the lines of traditional software development.

On review, the section “Code review at scale” explains the review points and how they handled rebases. It shows the use of review agents and the prompts that were probably given to them. I understood it as something close to the adversarial review above.

The article ends with Lessons learned, which mention the importance of phasing and a clear goal, and the importance of E2E tests.

In both migrations, one person did this much work, and the way of working seems to depend heavily on expert knowledge and experience. So at this stage, it seems fair to say that AI is boosting individual ability. I want to keep watching how the gap in output between individuals changes. When I think that a time may come when expertise is no longer a competitive advantage, it is scary.

Anthropic’s Code Review: Minimizing the Human Bottleneck

How the Claude Code team does code review. It is an article from October about how the Claude Code team does code review. About 90% of Claude Code’s code is written by Claude, and the number of daily merges in the monorepo grew from about 2,500 in May to over 30,000 in September. The goal is “merge most PRs with at most one human, without increasing reverts or incidents.” The metrics are the number of approvals per PR and human waiting time, and the guardrail is the revert and incident rate.

The article organizes the role of code review into two: finding problems, and building a shared understanding among team members. The basic idea is that human time and attention are precious resources, and the process is designed to minimize their use. This is something I have been struggling with lately, too.

On the first role, the article first shows how changes to Claude Code are merged and released. What differs from traditional software development is an automatic AI review similar to adversarial review, a separate AI review dedicated to security, and a mechanism called risk-based approval that decides whether a human review is needed. Billing, the auto-update mechanism, and CI are given as examples of risky areas. I guess this mechanism is built from a combination of rules and an LLM. Some problems still slip through these mechanisms. In that case, the on-call person handles the rollback, thinks about which layer should have caught the problem, and feeds it back into CLAUDE.md, skills, and the evaluation set to improve. The review and fix loop also has a limit: after three rounds, the reviewer asks the author to rethink the approach. One PR went through more than 50 rounds of review and fixes overnight, and about 60% of what the reviewer pointed out was about code that was there from the start.

On the second role, the team thinks that nobody needs to understand the intent of the whole codebase, and the Claude Code team is in that state. When they want to know how something works, they just ask Claude. Many industries and projects cannot accept this, but I think that deciding on a risk you can take case by case, and taking a bold approach like this, leads to a large productivity gain.

From the viewpoint of agent configuration, I was curious about how they split the scope of review. Splitting by perspective works better, as with separating normal review from security review, and quality drops when the scope gets too large. I feel this myself, too. But the article does not say why they split or how they cut it. This takes an engineer’s experience, and it seems we can only revisit it as models improve.

Summary

What the three had in common was: separating implementation and review into different contexts, preparing criteria for judging E2E execution results, and having humans own the goal, the phase boundaries, and high-risk decisions. And at Anthropic, they did not leave failures as they were; they fed them back into instructions and evaluation sets and fixed the loop itself. I felt this is the most important part.

At the scale of running agents in parallel, the human’s job shifts from writing code to designing and fixing the agents’ loops. This needs not only technical skill but also the ability to turn a feeling that “something is off” into instructions and mechanisms. Even if I call it verbalization, it is closer to the ability to bring out the tacit knowledge in my head. It looks like a multiplication: expertise is the source of the feeling, and verbalization turns it into a mechanism. I feel the way I use my head has changed a lot toward the meta side, and without this, it seems hard to make full use of agents.

As for how many people can think at the meta level: first, LLMs will evolve to be closer to human economic activity (as long as people stay in control), so I think they will develop toward asking back questions and offering templates that help with meta thinking. In addition, not every developer needs this ability. If many people who can build simple apps appear, there will be many workplaces where productivity goes up. Even in those places, I think people with expertise, verbalization skill, and meta thinking skill will still be more productive. And in large, parallel development like what we saw here, where the ability to fix loops is needed, I think these skills will be essential. I tried to think calmly, but some wishful thinking may be mixed in.

By the way, once you start thinking about fixing the loop that fixes the loop, the same question comes back every time the level of abstraction goes up one step. My own infinite loop does not seem to end soon.

Comments (0)

No comments yet. Be the first!