The score was not the most interesting result

The String Calculator was never the object of this experiment. It was a small Java kata often used to practice TDD, giving a coding agent a concrete reason to use Seed4J CLI.

The object was the interaction among three parts: the agent interpreting the goal and choosing capabilities, a skill installed in the repository guiding how it should discover, plan and verify, and Seed4J CLI validating and materializing the selected transformations.

In From an Empty Repository to a Java Kata, One Module at a Time, I followed that interaction through one complete Codex run, from an empty repository to a Java and Maven foundation and then to the kata implementation.

I had also repeated the same task five more times, changing which model powered Codex or how much reasoning effort it used. All six runs started from the same commit and kata specification, received the same prompt and used Seed4J CLI v0.0.4 with Seed4J 2.2.0. All six produced a working String Calculator implementation, passed the build included in the project and passed the same independent tests covering the kata requirements.

I initially evaluated them with a shared rubric. It awarded up to 100 points across Seed4J effectiveness, required behavior, tests and design. The rubric was useful for comparing the runs against the same criteria, but it reduced each execution to a score.

Comparing the commands and messages recorded during the six runs did not change the subject of the experiment. It made the different paths through the same interaction visible.

One run showed the workflow in action. Six runs showed how differently agents navigated that workflow.

What I mean by a decision trajectory

I am not trying to infer the model’s private reasoning. The recorded commands and messages do not expose it, and I do not need it for this comparison.

There is already a lot I can observe:

• which commands a run executes before changing the repository;
• which modules it inspects as candidates;
• which inspected modules it leaves out;
• whether it creates one complete plan or several smaller plans;
• how tool or environment feedback changes the next action;
• which capabilities it delegates to Seed4J;
• what it verifies after generation.

I think of that visible sequence as a decision trajectory.

The six runs used three models: Sol, Terra and Luna. Four used Sol at low, medium, high and xhigh reasoning effort; the other two used Terra and Luna at xhigh.

Three paths through the same CLI

The three xhigh runs provide the cleanest narrative comparison because the reasoning effort stays fixed while the selected model changes.

In the detailed evaluation, Sol xhigh scored 98/100, Luna xhigh 95/100 and Terra xhigh 91/100. All three earned the full 30 points for the required String Calculator behavior. Their remaining differences came from Seed4J usage, tests and design.

The behavior scores confirm a common functional baseline. They do not describe the different paths each run took through the interaction among agent, skill and CLI.

Sol xhigh anticipated the foundation

The Sol xhigh run inspected init, maven-java, maven-wrapper and jacoco-with-min-coverage-check, then placed all four modules into one apply-set plan.

Its visible trajectory looked roughly like this:

goal → candidate capabilities → one composition → plan → execution

This run anticipated that the project should have its own Maven launcher and an enforced coverage gate before attempting the kata implementation.

It also exposed an important division of responsibility. The run requested the modules in this order:

init
maven-java
maven-wrapper
jacoco-with-min-coverage-check

Seed4J executed them in this order:

init
maven-java
jacoco-with-min-coverage-check
maven-wrapper

The run chose the capabilities. Seed4J validated the composition and resolved how to materialize it.

Terra xhigh adapted to feedback

The Terra xhigh run began with a smaller composition: init and maven-java. It planned those modules, applied them and inspected the generated project.

Then it tried to use the global Maven command. The environment returned mvn: command not found.

The next visible message said:

The environment has Java 25, but does not have mvn.

That sentence is my English translation of the original Brazilian Portuguese message in the Terra run record.

The run then inspected maven-wrapper, generated an individual plan, applied the module and continued through ./mvnw.

Its trajectory was different:

initial composition → environment feedback → new capability → plan → execution

Calling this merely a mistake would hide the useful part. In this run, a missing environmental capability produced feedback, and that feedback changed the composition. The final project still recorded the wrapper through the same Seed4J history and commit mechanism.

Luna xhigh explored and kept the scaffold small

The Luna xhigh run inspected a broader group of candidates. In addition to init, maven-java and apply-set, it inspected java-base and spring-boot.

After that exploration, its visible message stated that the specification required incremental TDD in Java but did not require a framework or application structure. The run chose a minimal Maven project with JUnit 5 and left java-base and spring-boot out of the composition.

It later inspected maven-wrapper and planned the final set:

candidate exploration → minimal boundary → composition → plan → execution

This is not evidence that Luna generally explores more or that it is inherently more disciplined about scope. It is evidence that this particular run inspected two plausible capabilities and did not apply them.

The other runs left signals too

The remaining Sol runs reinforce the point that a score or final file tree does not describe the complete interaction.

This table does not rank the trajectories. It gives me a vocabulary for comparing them: discovery breadth, candidate rejection, planning granularity, proactive composition, reaction to feedback and delegation to the tool.

The skill constrained the protocol, not every decision

All six runs had the same Seed4J CLI skill installed in the repository.

The skill defines an operating protocol:

1. discover the CLI, runtime and catalog;
2. infer candidate modules from the task;
3. inspect the active help for those modules;
4. plan before mutation;
5. execute the validated composition;
6. verify the generated result and Git state.

That protocol explains part of the consistency across the runs. Every one of them discovered the runtime and catalog. Every one produced a plan before applying modules. Every successful module left history and a commit.

But the skill did not prescribe the exact module set for the kata. It did not say that JaCoCo was mandatory. It did not tell the run whether to inspect Spring Boot, when to add the Maven Wrapper or whether to construct one large plan.

That left a meaningful decision surface for each run:

• the run selected the capabilities it wanted;
• the skill guided how to interact with the tool safely;
• Seed4J validated dependencies and parameters, calculated execution order, changed files, updated module history and created commits.

This separation is more interesting to me than trying to put every correct decision into the skill.

The tool reduced the space of decisions that had to be right

Without a project generator, a coding run building the same foundation might need to choose Maven plugin versions, write the POM, create the wrapper, configure JaCoCo, place files correctly, preserve project history and decide how to divide infrastructure changes into commits.

With Seed4J, the request can be closer to a set of capabilities:

Java with Maven
Maven Wrapper
coverage gate

The tool then turns that explicit intent into deterministic transformations.

This does not make module selection irrelevant. A run can still omit a useful capability, add one the task does not need or choose parameters poorly. Seed4J also cannot guarantee that the kata implementation itself will be well designed.

What it can do is narrow the area in which those choices operate. Once a valid capability is selected, the run does not also need to reproduce all of the configuration details behind it from memory.

The models can take different routes through discovery and planning without every difference becoming a different manually written build configuration.

This connects to how I think about skills and loops

In When Skill Evolution Means Removing Instructions, I argued that deterministic knowledge should move into deterministic mechanisms whenever possible. A skill can become smaller when a test, hook or tool can enforce what used to depend on an instruction.

In I Had Already Built Three Agentic Loops Without Naming Them, I described autonomy becoming safer when feedback and exit conditions live in the environment instead of depending on the model remembering them.

Seed4J gives me another concrete example of both ideas.

The skill does not contain a static catalog or teach the model how to write every generated file. It sends the run to the active CLI. The plan provides feedback before mutation. The runtime validates the composition. The module history and Git commits make the result inspectable afterward.

The model still decides. But it decides inside a workflow with explicit feedback and deterministic boundaries.

What this experiment does not show

This was one kata, one prompt and one run for each model and effort configuration, all executed on the same host. Each run took place in a separate chat, and execution time was not part of the comparison. The records also differ in format and in what they omit, which limits direct comparisons of presentation and completeness.

There was no control group implementing the kata without Seed4J. I therefore cannot claim that Seed4J made the runs faster, cheaper or more correct than another approach.

Most importantly, these trajectories do not establish stable model personalities. I can say that the Terra xhigh run added the wrapper reactively. I cannot conclude from one run that Terra is a reactive model. I can say that the Sol xhigh run anticipated the coverage gate. I cannot conclude that Sol always plans infrastructure better.

The detailed experiment report keeps the full protocol, commands, scores and limitations.

A principle I want to test again

These six runs do not prove a general rule about agent tools. They make one design principle concrete enough for me to keep testing.

Different runs can explore different candidates, compose at different moments and react differently to feedback. A tool does not necessarily need to normalize all of that behavior. It can preserve room for judgment while making dependencies, ordering, parameter resolution, history and mutation more predictable.

A good agent tool may not need to eliminate model variability. It may need to constrain where that variability can cause damage.

If this experiment made you curious about the approach, consider giving Seed4J and Seed4J CLI a star 🌟 on GitHub. It helps more people discover the projects and follow their evolution.

References

• Seed4J
• Seed4J CLI
• Seed4J CLI String Calculator Kata
• Detailed experiment report
• Sol xhigh run record
• Terra xhigh run record
• Luna xhigh run record
• OpenAI model catalog

A small project to learn Seed4J CLI

I wanted a small project to learn how to use Seed4J CLI, but with a slightly different constraint: I wanted to learn it the same way I expect to use it in real projects, together with a coding agent.

Seed4J is a modular code generator. Instead of generating one fixed project, it applies named transformations, called modules, that add or configure one capability at a time.

The Seed4J CLI makes that approach available from the command line.

So I created a repository around the classic String Calculator kata, using the specification from the Coding Kata Catalog. It is a small testing exercise in which an add function starts with simple string inputs and gradually supports more rules, such as multiple numbers and custom delimiters.

The Seed4J CLI can also install a skill directly in the repository where it will be used. That skill teaches the coding agent how to discover and operate the CLI. The skill guides the agent; the Seed4J modules are what transform the project.

The starting point is intentionally minimal: a specification for the kata and the Seed4J CLI skill installed locally for the agent.

The original prompt was written in Brazilian Portuguese:

implement o kata utilizando o seed4j cli tool já instalado como apoio.

For readability, I translated it to English:

Implement the kata with support from the installed Seed4J CLI tool.

From there, the agent has to inspect the active CLI and decide how to compose the project.

I later repeated the same task across six model and reasoning configurations. Their differences are documented in the detailed experiment report linked below. To keep this post focused, it follows the string-calculator-sol-xhigh run, which used the Sol model with xhigh reasoning effort.

Let the agent discover the CLI

The agent starts by discovering the command-line client, the Seed4J version behind it, and the modules available to it:

seed4j --version && seed4j --help
seed4j list

The command-line client and the Seed4J generator it invokes are versioned separately. This run used Seed4J CLI v0.0.4 with Seed4J 2.2.0.

From the available modules, the agent identifies four capabilities that seem useful and inspects their parameters:

seed4j apply-set --help
seed4j apply init --help
seed4j apply maven-java --help
seed4j apply maven-wrapper --help
seed4j apply jacoco-with-min-coverage-check --help

The module names describe four separate transformations. Each name below links to the commit Seed4J created for that transformation:

• init establishes the common project foundation: project metadata, repository conventions, development tooling, and a record of the Seed4J modules applied. It does not create the Java application.
• maven-java adds the Java and Maven build, including the testing dependencies.
• maven-wrapper adds a Maven launcher to the project, so the build does not depend on a separately installed Maven version.
• jacoco-with-min-coverage-check turns code coverage into a build requirement.

The apply-set command composes several modules and resolves a valid execution order. Before changing the repository, the agent uses --plan to preview the resolved modules and parameters without applying them:

seed4j apply-set init maven-java maven-wrapper jacoco-with-min-coverage-check --plan --project-path . --project-name 'String Calculator Kata' --base-name stringCalculator --package-name com.renanfranca.stringcalculator --node-package-manager npm --end-of-line lf --indent-size 2

The plan is valid and Seed4J resolves the execution order as:

init
maven-java
jacoco-with-min-coverage-check
maven-wrapper

Notice that this is not exactly the order requested by the agent. It placed maven-wrapper before jacoco-with-min-coverage-check. The agent’s order was valid: both modules can be applied after maven-java, and neither depends on the other. Seed4J does not preserve the requested order, however. Because both modules are at the same dependency level, Seed4J 2.2.0 sorts them lexically by slug, placing jacoco-with-min-coverage-check before maven-wrapper.

After reviewing the plan, the agent applies exactly the same composition without --plan.

Seed4J applies the modules sequentially and creates one commit for each successful module.

Once Seed4J finishes, the repository has a common project foundation, a Java 25 Maven build with testing libraries, an enforced coverage threshold, and its own Maven launcher. Seed4J stops at that foundation; it does not generate the String Calculator. From there, the agent implements the kata in one production class and one test class. Running ./mvnw verify executes 13 behavior tests and enforces 100% line and branch coverage with JaCoCo.

What interested me was not the generated files alone. It was watching the agent discover the available capabilities, select the modules it needed, and let Seed4J resolve and record the composition one module at a time.

The kata is not the thing I am trying to learn

I chose a familiar kata precisely because its implementation was not the variable I wanted to study. It gives me a small, controlled problem where I can watch an AI agent discover Seed4J, choose modules, inspect their parameters, preview the resulting composition, and use those modules to turn an almost empty repository into a Java project.

Instead of asking the agent to manually recreate all the project setup, I can watch it decide which capabilities it needs and let Seed4J apply the corresponding transformations.

The repository records this experiment, including the commands executed by the agents, the modules they selected, the resulting projects, and their Seed4J module history.

What I can observe here is how the agents use Seed4J under this setup. Because every run uses Seed4J, this experiment does not tell me whether Seed4J makes an agent faster, cheaper, or better than another scaffolding approach.

A small training ground

For me, this makes the kata a small training ground for a larger question: how should I use Seed4J together with AI coding agents in real projects?

I do not think this kata answers that question. It gives me a controlled place to observe the interaction, learn what information the agent needs, and make the next experiment more concrete.

If this experiment made you curious about the approach, consider giving Seed4J and Seed4J CLI a star 🌟 on GitHub. It helps more people discover the projects and follow their evolution.

Reproduction note

Seed4J CLI itself requires Java 25 or newer on your PATH, and its npm launcher requires Node.js 22 or newer. With those prerequisites available, install the CLI and verify the command:

npm install -g seed4j-cli
seed4j --version

Then, install the optional skill that guides the coding agent by running this command from the kata repository:

seed4j skill install

I used Codex in the ChatGPT desktop app with Full Access enabled so Seed4J could write to .git and create a separate commit for each module without approval interruptions. In Codex CLI, the equivalent is running codex --yolo. Because that bypasses approvals and sandboxing, it should only be used in a controlled or externally isolated environment.

References

• Seed4J
• Seed4J CLI
• Coding Kata Catalog
• Seed4J CLI String Calculator Kata
• Sol xhigh implementation
• Detailed experiment report
• Codex sandbox and permissions

Two papers, one familiar question

I keep having this strange experience lately: I spend time experimenting with something in a very small, practical way, and then a paper appears exploring a remarkably similar question at a much larger scale.

Two recent papers made that happen again.

The first is “Evaluating Skills, Not Just Agents”, which introduces ACES and Skill Lift: execute the same task with and without a skill and measure the difference.

That connects directly with what I was exploring in skill-eval.

But my main takeaway from that experiment was actually about the limits of A/B testing.

An A/B comparison gives evidence for that particular model, task, context and harness. It does not prove that a skill is universally good. Change the model and the skill may become less useful, unnecessary, or even harmful.

Knowledge does not have to become instructions

Then I read Google Research’s “WikiSkill: Compiling Agent Experience into Persistent Knowledge for Skill Evolution”.

And another piece clicked into place.

WikiSkill separates raw experience, accumulated knowledge and the actual executable skill. Experience can keep accumulating outside the skill, while proposed skill changes are validated before being promoted.

Even more interestingly, the paper shows cases where low-level workarounds learned for one model transfer poorly to a stronger model. Instructions that helped the weaker model could constrain the stronger one or cause redundant tool calls.

That is very close to another experiment I did inside skill-eval.

Evolving a skill by making it smaller

I tried to “evolve” my implement-execplan skill not by adding more instructions, but by removing them.

The original baseline had 228 lines. The adaptive version had 165.

Instead of requiring a large fixed plan structure and continuous updates, the newer version relies more on the model’s own capabilities: fewer mandatory sections, conditional information only when useful, fewer plan updates and less routine bookkeeping.

The skill is here:

implement-execplan/SKILL.md

And the case study is here:

skill-eval: implement-execplan case study

In the first valid comparison, both versions passed the same quality gates, while the smaller version used about 18% fewer candidate input + output tokens.

That is only one observation, not proof that the new version is universally better. The replication was incomplete, and I deliberately documented that limitation.

But it changed how I think about “skill evolution”.

Evolution does not necessarily mean accumulating more rules.

Sometimes the model improved and part of the harness became obsolete.

Sometimes something that used to require an instruction can become a deterministic test, hook or static check and disappear from the skill entirely.

This is also a continuation of something I noticed while gradually giving TDD to an agent. In When I Entrusted TDD to an AI Agent, some architectural guidance became much more reliable once it stopped being only an instruction and became executable feedback in the repository.

And sometimes a failure is worth remembering, but not yet worth turning into an instruction. This is where the WikiSkill separation between experience, persistent knowledge and executable skills makes a lot of sense to me.

I prefer to discover the workflow first

This also changed how I create skills.

I prefer to experience the problem first, work with the model manually, adjust the workflow until I can make it work reliably, and only then consolidate what I learned into a skill.

My latest implement-approved-plan skill came from exactly that process:

implement-approved-plan/SKILL.md

This one is interesting because the skill itself coordinates a multi-chat workflow inside ChatGPT Desktop.

A dedicated Coordinator keeps the approved plan and controls a set of specialized chats. Each role is created once and then reused throughout the implementation: an Implementer works through TDD, a Committer owns commits, a Validator runs the complete quality gates, a Habit Curator handles mechanically detected design findings, and an independent Structural Reviewer looks for design problems after the implementation is already green.

The specialization is not only about giving each chat a different responsibility. The skill also assigns different models to different roles. The Coordinator, Implementer, Habit Curator, and Structural Reviewer run on gpt-5.6-sol at xhigh; the Committer runs on gpt-5.6-terra at xhigh; and the Validator runs on gpt-5.6-luna at xhigh.

That routing makes the workflow more economical. Sol is reserved for open-ended implementation, orchestration, and design judgment; Terra handles commit work, which is narrower but still requires judgment about repository conventions; and Luna runs the mostly deterministic validation gates. The point is not merely to create specialists, but to avoid using the flagship model where a more economical model is sufficient.

Conceptually, it looks roughly like this:

Approved Plan
     ↓
Coordinator
     ↓
Implementer
     ↓
Coordinator
     ↓
Committer
     ↓
Coordinator
     ↓
Validator
     ↓
Coordinator
     ↓
Habit Curator
     ↓
Coordinator
     ↓
Structural Reviewer
     ↓
Final Validation
     ↓
PR + CI
     ↓
Ready for Merge

The interesting part for me is that these chats do not freely coordinate with each other. The Coordinator is the only one talking to the specialists, and access to the same working tree is serialized through a persistent workflow ledger.

That ledger works almost like a small state machine. It records the current phase, specialist chat IDs, commits, validation gates, Habit state, pull request and CI evidence. It rejects invalid transitions, concurrent ownership of the checkout, premature pull requests and cleanup before GitHub actually confirms that the PR was merged.

The shape is more elaborate than the three nested loops I described in I Had Already Built Three Agentic Loops Without Naming Them, but the underlying idea feels similar: autonomy becomes safer when feedback and exit conditions live in the workflow instead of depending on the model remembering everything.

So the skill is not trying to make one giant agent smarter. It is encoding a workflow I had already been performing manually, while moving as much coordination and validation as possible into deterministic machinery around the model.

PR that introduced it:

renanfranca/codex-skills#7

And instead of stopping when the SKILL.md looked good, I exercised the workflow against a public fixture and a real pull request:

implement-approved-plan-fixture

implement-approved-plan-fixture#2

My current mental model

So my current mental model is becoming something like this:

Experience the problem.

Make the workflow work manually.

Turn deterministic knowledge into deterministic mechanisms.

Keep uncertain experience outside the skill until it earns its place.

Add instructions only for what the model still needs.

And when the model changes, question those instructions again.

No magic skill. No assumption that more instructions mean a better agent.

Just a workflow that keeps adapting as both the model and my understanding of the problem change.

I was already working with loops

After reading Laurie Voss’s article What the hell is a loop, anyway?, I realized that I had naturally ended up with three layers of loops in my workflow with coding agents.

I had not deliberately designed something I called a loop architecture.

I had been trying to solve practical problems.

How could I give an agent enough autonomy to implement a complete feature without losing the destination?

How could I divide a large implementation into observable units of progress?

How could I ensure that the code evolved through behavior while continuing to respect the architecture?

The solutions I adopted gradually became three nested feedback loops.

I just did not have a name for them.

These are loops inside one implementation

My three layers do not map perfectly to the four loop architectures Laurie describes.

I am not running an autonomous software factory that chooses work from a backlog, ships it, monitors production, and starts again.

The loops I see are inside the implementation of a single change.

The outer loop follows the complete execution plan.

Inside it, another loop follows the milestones.

Inside each milestone, TDD can generate several smaller red-green-refactor cycles.

Each layer operates at a different level of abstraction, receives different feedback, and has a different exit condition.

The outer loop: the execution plan

The outermost layer is the execution plan.

As I described in When Plan Mode Was No Longer Enough, the execution plan gives the agent more than a list of tasks.

It describes:

• why the change is needed;
• the current context;
• the desired final behavior;
• relevant constraints and decisions;
• the implementation milestones;
• the validation strategy;
• the final acceptance criteria.

This loop begins with the current state of the repository and a desired destination.

The agent implements, validates, updates the plan with what it discovers, and continues until the complete plan satisfies its final acceptance criteria.

The important part is that the loop does not end merely because the agent says that it has finished.

It ends when the observable result matches what the plan defined as complete.

The execution plan therefore protects the destination.

Local implementation decisions can change. A milestone may reveal that an earlier assumption was incomplete. The agent may discover a better design while practising TDD.

The plan can evolve with those discoveries, but the final outcome must remain explicit.

The middle loop: the milestones

Inside the execution plan, there are milestones.

Each milestone has its own scope, validation commands, expected result, and acceptance criteria.

A milestone creates a smaller convergence point inside the larger implementation.

Instead of asking only:

Has the complete feature been implemented?

The plan can repeatedly ask:

Has this meaningful part of the feature reached the state it promised?

This distinction matters even though I no longer stop the agent after every milestone.

Earlier in my workflow, each milestone was also a human approval gate. I reviewed the implementation and explicitly authorized the next step.

Today, I usually allow the agent to implement the complete execution plan autonomously.

However, removing the human interruption did not remove the milestone loop.

The agent still needs to complete the milestone, run its validations, verify its acceptance criteria, record relevant decisions, and only then move forward.

The milestone remains a checkpoint even when I am not standing at the checkpoint.

It protects intermediate progress.

Without this layer, the agent could perform many technically valid operations without demonstrating that the implementation is moving through meaningful states toward the final result.

The inner loop: TDD and executable architecture

Inside the milestones is the smallest and most active layer.

The TDD loop chooses the next observable behavior, writes a failing test, implements the minimum code necessary to make it pass, evaluates the opportunity for refactoring, and continues while the suite remains green.

One milestone can generate many of these cycles.

As I described in When I Entrusted TDD to an AI Agent, I eventually allowed Codex CLI to run these cycles autonomously through my tdd-behavior-autonomous-quiet skill.

But TDD alone was not enough.

An agent can write passing tests while moving behavior into the wrong architectural layer. It can also create tests that merely reproduce the structure of the production code.

For this reason, the loop also receives feedback from architectural tests such as HexagonalArchTest.

These tests make architectural boundaries executable.

When the agent introduces an invalid dependency, the repository rejects the design.

The failure creates another correction cycle:

1. the behavioral test may already be green;
2. an architectural rule fails;
3. the agent changes the design;
4. both behavioral and architectural tests run again;
5. the implementation continues only when both forms of feedback agree.

Vertical checkpoints add another feedback signal.

After a small number of TDD cycles, the agent must validate the feature through its public path. This prevents locally successful cycles from drifting into what I called the implementation details loop.

The inner layer therefore does more than ask whether the latest unit test passes.

It asks whether the behavior works, whether the design respects the architecture, and whether the accumulated implementation still produces progress through the feature’s public path.

Each layer asks a different question

Looking at the workflow as nested loops helped me understand why I need all three layers.

The execution plan asks:

Are we reaching the final outcome?

The milestone asks:

Have we completed this meaningful stage of the implementation?

The TDD and architecture loop asks:

Does the next behavior work without compromising the design?

None of these questions replaces the others.

A green TDD cycle does not prove that a milestone is complete.

A completed milestone does not prove that the entire feature satisfies its acceptance criteria.

A completed execution plan does not prove that every internal design decision is good.

Each loop provides feedback at the scale where that feedback is useful.

Autonomy depends on where feedback lives

Previously, much of this feedback lived in my interventions.

I inspected each step, questioned the direction, approved the next behavior, and decided when the agent could continue.

As I removed confirmation gates, I needed to move that judgment into executable constraints.

The execution plan made the destination explicit.

The milestone criteria made intermediate progress observable.

Behavior tests made functionality executable.

Architectural tests made design boundaries executable.

Vertical checkpoints made progress through the public path executable.

The agent did not become more autonomous because I stopped caring about the process.

It became more autonomous because the process could increasingly evaluate itself without waiting for me to repeat the same instructions.

The loops still end with me

Even with these layers, I remain outside the complete implementation loop.

When the agent finishes, I review the result as a pull request.

I read the behavior tests, follow the production code that makes them pass, inspect the architectural decisions, refactor where necessary, and decide whether the change belongs in the codebase.

The loops reduce the number of routine decisions that depend on my immediate participation.

They do not transfer responsibility for the final code.

For now, the simplest names I have are:

• the execution plan loop;
• the milestone loop;
• the TDD and architecture loop.

Perhaps better names will emerge.

What matters to me is that I can finally see the structure that was already there.

I was not merely giving an agent a large prompt and waiting for code.

I was stacking feedback loops, each with its own scale, evidence, and exit condition.

I just did not know that this was what I was doing.

Keeping the tests under my control

When I first started using AI agents for software development, I kept a clear division of responsibilities.

I wrote the tests.

The agent implemented the production code and helped me refactor it.

At the time, I was using Aider through its CLI.

This arrangement allowed me to benefit from the agent without giving up control of the TDD process. I still chose the next behavior, wrote the failing test, watched it fail, and decided when the implementation was ready to be refactored.

More importantly, it kept my anxiety under control.

As I described in For Me, TDD Is Therapy, TDD gives me a systematic way to move through the uncertainty of software development. Delegating production code while retaining the tests allowed me to preserve that process.

Then I watched a presentation that changed how I thought about this division.

Seeing an LLM practise TDD

The presentation was Lada Kesseler’s excellent Augmented Coding: Mapping the Uncharted Territory.

For the first time, I saw the possibility of allowing an LLM to write both the tests and the production code while still following the TDD flow.

Until then, I had assumed that keeping control of TDD meant keeping control of the tests.

Lada’s presentation showed me another possibility: perhaps I could delegate the entire red-green-refactor cycle while still making the process visible and inspectable.

I was so excited that I sent Lada Kesseler a message on LinkedIn to thank her.

She very kindly sent me the prompt she had used to guide the LLM through TDD.

Reusable agent skills, at least in the form in which I use them today, did not exist yet. Even so, her presentation and prompt were already exploring the problems that skills would later help me address.

My first TDD skill

When skills emerged, I used Lada’s prompt as the foundation for my first tdd skill.

Sorry, Lada, if I somehow ruined your original prompt 😅.

I started using the skill with Codex CLI to implement new features inside existing business domains.

The process was extremely interactive.

The agent would:

1. write a comment describing the intention of the smallest test it wanted to implement;
2. wait for my approval;
3. implement the test and explain how it expected the test to fail;
4. wait for approval again;
5. implement the minimum production code necessary to make the test pass;
6. wait before refactoring or moving to the next behavior.

It was beautiful to watch.

The tests led the implementation. The expected failure was made explicit. Production code was added incrementally. Refactoring happened only while the test suite was green.

Most importantly, I could see the process.

I was no longer looking only at a final diff containing tests and production code. I could follow the sequence through which the design emerged.

However, I gradually started to notice another problem.

My need for control was forcing me to use the agent with the handbrake on.

Removing confirmations from inside the cycle

My next experiment was tdd-strict-cycle-confirmation.

Instead of requiring approval at several boundaries inside the same TDD cycle, the agent could complete red, green, and refactor before returning control to me.

The practical effect was simple.

I approved the next behavior. The agent wrote the failing test, implemented the minimum code, evaluated the opportunity for refactoring, reran the relevant tests, and then stopped before beginning another cycle.

This made the workflow considerably smoother.

I still had frequent opportunities to intervene, but I no longer needed to authorize every mechanical transition. The feedback loop remained small enough for me to understand what had happened without unnecessarily interrupting the agent.

For a while, this felt like the right balance.

Then I wanted to know what would happen if I removed even that confirmation gate.

Letting the agent finish the feature

I created another variation called tdd-strict-autonomous.

This version no longer stopped after every completed cycle. It continued automatically through red, green, and refactor, pausing only when it encountered an exceptional situation such as ambiguous behavior, an unexpected failure, an architectural decision, or an environment problem.

At the time, I was trying to implement Registration of custom seed4j instances.

The feature would allow the Seed4j CLI to register additional Seed4j module providers and discover their modules at runtime.

Even I did not know exactly what the technical solution should look like.

That uncertainty was part of the experiment. Instead of directing every design decision, I wanted to observe what the agent would discover through TDD.

Unfortunately, the experiment exposed a serious problem.

The implementation details loop

I called the problem the implementation details loop.

The agent continued taking small steps. Each individual step appeared reasonable. Tests were being added, production code was changing, and the suite kept moving forward.

But the implementation was no longer moving meaningfully toward the feature.

The agent had become absorbed in the internal structure it was creating.

A new class encouraged a new test class. That test exposed another internal detail. Supporting that detail required another production abstraction, which then received its own tests.

The cycles were small, but the feature was disappearing from view.

When I inspected the Codex CLI session, I found so much information that I could not identify the exact moment when the implementation had lost its direction.

The problem was not a lack of activity.

The problem was that local progress no longer implied progress through the feature’s public path.

I abandoned the implementation and returned to tdd-strict-cycle-confirmation.

This time, I wanted to find the precise cycle in which the agent started following implementation details instead of behavior.

After finding it, I told Codex CLI that it was trapped in a loop and added this instruction to the TDD skills:

Run a vertical checkpoint at least every two cycles using the public path of the feature.

The agent could still work through small units, but it now had to regularly prove that those units composed into observable progress.

This improved the situation, but it did not solve everything.

Making the autonomous process quieter

The agent still produced a large volume of messages.

Detailed output had initially helped me trust the process. Once the agent became autonomous, however, the same volume made it harder to notice the information that actually mattered.

Routine file reads, expected failures, test commands, green transitions, and minor refactorings created a long stream of activity.

I created tdd-strict-autonomous-quiet to reduce that noise.

Quiet does not mean merely writing shorter messages.

It means emitting fewer messages.

The agent keeps ordinary cycle details internal and interrupts me only for meaningful events:

• an unexpected failure;
• a blocked environment;
• a public-path checkpoint failure;
• an architectural or public API decision;
• a significant refactoring;
• a final summary.

This made autonomous execution much easier to follow.

However, the quieter output also made another problem clearer.

The agent was still creating too many tests tied to the structure of the production code.

Tests should not mirror production topology

I frequently saw one test class for every production class.

When the implementation extracted a helper, resolver, parser, mapper, or adapter, the agent often created a corresponding test.

The final suite could have excellent coverage while still being fragile.

A refactoring that renamed, moved, split, or merged production classes could require several tests to change even when no observable behavior had changed.

I could correct the direction manually, but doing so repeatedly was becoming too much work.

The agent was failing to apply a concept presented in complementary ways in these two articles:

1. Test Contra-variance, by Uncle Bob;
2. Additional Testing After Refactoring, by Kent Beck.

My interpretation is that the shape of the tests should not follow the shape of the production code.

Production code may become more detailed as the design evolves. It may gain more classes, functions, modules, and internal abstractions.

The tests do not need to reproduce that topology.

They should remain attached to observable behavior and stable contracts.

A test is suspicious when it fails merely because the production code was internally reorganized while preserving the same behavior.

Architecture defines stable observation points

Before improving the TDD skill again, I realized that the agent was also failing to respect the chosen hexagonal architecture consistently.

This matters because architecture helps identify useful and stable observation points for tests.

A command-line feature can often be tested through the CLI boundary. An application behavior can be exercised through an application service. Domain rules can be verified through domain contracts.

Internal adapters and implementation helpers usually do not need tests merely because they exist.

For a practical explanation of this relationship, I recommend TDD in Hexagonal and Clean Architecture, by Valentina Jemuović.

Without enforceable architectural boundaries, an agent can gradually move behavior into the wrong layer while continuing to produce passing tests.

The tests may verify the code it created without protecting the design I intended.

Prompt instructions alone were not enough.

I needed the repository itself to reject architectural violations.

Making the architecture executable

I implemented architectural tests such as HexagonalArchTest.

These tests use ArchUnit to enforce rules including:

• domain code must not depend on infrastructure;
• application code must not depend on infrastructure adapters;
• primary adapters must not depend directly on secondary adapters;
• secondary adapters must not depend on the application layer;
• bounded contexts must communicate through the intended boundaries;
• domain ports must represent business capabilities;
• composition code must remain in the composition root.

This changed my relationship with the agent.

I no longer needed to rely only on a written instruction saying, “Respect the hexagonal architecture.”

The test suite could detect violations.

The architecture became executable feedback inside the same loop the agent was already following.

When an architectural test failed, the agent received concrete evidence that its current design had crossed a boundary.

From strict TDD to behavior focused TDD

The remaining task was to combine everything I had learned into another skill.

I created tdd-behavior-autonomous-quiet.

It preserves autonomous red-green-refactor cycles and quiet output, but it adds an important requirement: tests must follow observable behavior, public contracts, user journeys, or intentionally stable component APIs.

They must not follow production file and class structure.

Before creating a new test class, the agent must effectively answer three questions:

1. Which behavior is being specified?
2. Through which public or stable API is that behavior observed?
3. Why is no existing behavior test suite the appropriate home for it?

Extracting a new internal class does not automatically justify creating a new test class.

Moving production code does not automatically justify moving tests.

A lower-level test is appropriate only when the component has a stable API that is independently meaningful to a caller.

The agent should prefer the highest useful observation point that still provides clear and sufficiently fast feedback.

This was the missing piece.

Combining the skill with execution plans

Today, I use tdd-behavior-autonomous-quiet to implement execution plans created through the process I described in When Plan Mode Was No Longer Enough.

Previously, I asked the agent to implement one milestone at a time. I reviewed the result and explicitly authorized the next milestone.

I no longer do that.

I realized that a milestone does not always correspond to a complete feature or even a complete design decision.

Sometimes I reviewed an intermediate state and corrected something that would naturally have been resolved in a later milestone. My intervention was based on incomplete information.

Now I wait for the entire execution plan to be implemented.

The execution plan gives the agent the destination, context, constraints, validation strategy, and important decisions.

The TDD skill controls how the implementation moves toward that destination.

The architectural tests protect the boundaries of the system.

The vertical checkpoints verify that local cycles continue producing progress through the public path.

Together, these elements allow the agent to work autonomously without reducing the process to “generate everything and add tests afterwards.”

Reviewing the result as a pull request

The result is usually well designed code with tests that genuinely verify behavior.

I still find opportunities for refactoring and improvement.

But the experience now feels like reviewing a very well written pull request.

I begin with the first behavior test.

I understand what it protects.

Then I examine the production code that makes it pass.

I continue following that relationship between behavior and implementation until I complete the review.

A practical example is the pull request enhance apply plan dependency status.

The first commit contains exactly what the agent developed, without any intervention from me.

The remaining commits show the adjustments I made during my review.

That distinction is important because it makes the result inspectable.

It shows what autonomous implementation produced and where my judgment still changed the code.

Autonomy introduced a different waiting problem

Waiting between three and six minutes for the agent to deliver the implementation still bothers me.

The agent writes quickly enough that I cannot meaningfully follow every operation in real time.

At the same time, it is slow enough to leave me waiting and disconnected from the problem solving process.

Ideally, the interaction would respond within the precious 400 milliseconds, preserving the feeling of a continuous workflow.

Instead, I delegate the work, wait, and later receive an implementation that I need to load into my mind.

This creates a different kind of cognitive cost.

I am not a multitasker

I am not a multitasker.

I can perform tasks sequentially, and TDD supported this extremely well.

After completing a red-green-refactor cycle, I had a natural stopping point. If I was interrupted, I knew that the suite was green and the current behavior had been completed.

When I returned, the next failing test or pending behavior told me where to continue.

Reviewing an autonomous implementation is different.

If I am interrupted in the middle of the review, I need to mark which test I have already examined and which one I should read next.

I also carry a greater cognitive load because I did not participate directly in each problem solving step.

The final code may be clear, but the path that produced it is not fixed in my mind in the same way as code I developed myself.

The agent has reduced the effort required to produce the implementation.

It has not eliminated the effort required for me to understand and assume responsibility for it.

Answering my previous questions

In For Me, TDD Is Therapy, I asked three questions.

I can now answer two of them.

Am I limiting the AI agent by requiring it to work through TDD?

I reduced my need for control as much as I currently consider responsible.

I no longer write the tests myself. I do not approve every test, every green implementation, every refactoring, every cycle, or every milestone.

I entrust the TDD process to the agent until the complete execution plan has been implemented or a meaningful exception requires my attention.

The constraints remain strict, but routine control no longer depends on me.

How can I reproduce the TDD feedback loop when working with an AI agent?

I moved toward it gradually.

Initially, every transition was visible and required confirmation. Later, I approved only complete cycles. Then I allowed cycles to continue autonomously.

Today, most routine steps remain internal, but the agent surfaces relevant decisions, failures, checkpoints, and refactorings.

I no longer experience the feedback loop by personally performing each step.

Instead, I make the loop executable through the skill, the test suite, the architectural rules, and the public path checkpoints.

That is not the same experience as practising TDD manually, but it preserves the properties I care about.

The third question remains more difficult.

Does the agent genuinely practise TDD, or does it develop the solution first and create the tests afterwards?

My friend Victor Perone introduced me to a few machine learning concepts.

That alone was enough for me to realize how different this field is from my own.

To answer this question properly, I would need to investigate much more deeply what happens inside an LLM while it appears to perform a TDD cycle.

Anthropic itself is still investigating how language models internally reach their outputs, as shown in On the Biology of a Large Language Model and Verbalizable Representations Form a Global Workspace in Language Models.

For now, I can verify the external process.

I can require a failing behavior test before production code changes.

I can inspect the failure.

I can constrain the green implementation.

I can require refactoring only while the suite is green.

I can enforce architectural boundaries and repeatedly validate the feature through its public path.

I still cannot claim to know what happened internally before the agent displayed each step.

What I actually learned to trust

I did not learn to trust every line generated by an AI agent.

I learned to trust a process with executable constraints.

The agent owns the individual TDD cycles.

The execution plan preserves the destination.

Behavior tests protect what users and callers can observe.

Architectural tests protect the system’s boundaries.

Vertical checkpoints prevent small implementation steps from losing the feature.

I remain responsible for reviewing the result and deciding whether it belongs in the codebase.

Delegating TDD did not remove me from software development.

It changed where I participate.

The anxiety I recognized

While watching the podcast How Kent Beck Shapes the Software Engineering Industry, I heard Kent Beck describe the anxiety he felt while developing software and how Test-Driven Development helped him deal with it.

I immediately recognized that feeling.

I have felt it since my first contact with programming in 2006, during my Computer Science degree.

I never knew exactly where to begin an implementation.

The process was always different. It was highly intuitive and depended heavily on my mood, motivation, and confidence on that particular day.

Sometimes I would spend too much time thinking about the best place to start. Other times I would begin somewhere almost randomly and hope that the implementation would eventually reveal its shape.

There was no process I could trust.

When working code was not enough

The uncertainty was so intense that, when something I implemented worked on the first try, I did not trust it.

A successful result was not enough to convince me.

I needed to deliberately break the production code and repeat the manual test. Only after watching the behavior fail could I feel confident that my manual verification was actually exercising the change.

Otherwise, I was left with uncomfortable questions:

• Did the implementation really work?
• Was my manual test actually exercising the changed behavior?
• Did it work for the reason I expected?
• Was there some accidental condition making everything appear correct?

I did not have a reliable way to move from uncertainty to confidence.

For me, TDD is therapy

For me, TDD is therapy.

That is what I believe.

Not because TDD removes uncertainty from software development. It does not.

It gives me a systematic process for moving through that uncertainty.

I do not need to discover the entire implementation before I begin. I only need to identify the next observable behavior and express it as a test.

Then I watch it fail.

I implement enough to make it pass.

I refactor while keeping the behavior protected.

And then I repeat the cycle.

This process gives me somewhere to begin and tells me what to do next.

On my best days, it gives structure to my ideas.

On my worst days, it allows me to remain productive even when motivation and confidence are low.

That is my main reason for practising TDD.

The other benefits matter: executable documentation, regression protection, simpler designs, shorter feedback loops, and the confidence to refactor.

But my primary motivation is more personal.

TDD gives me a process I can follow when my intuition is not enough.

How I started practising TDD

I started deliberately practising TDD in December 2022 while contributing to the open source project now known as Seed4j.

In How did I earn money to fix an issue from an Open Source Software?, I described some of that first experience.

At the time, Seed4j was still called JHipster Lite.

That contribution also taught me an important distinction. I initially believed I was doing TDD, but I was actually writing tests before the implementation without necessarily allowing the tests to guide the design.

I was doing test first.

Learning that distinction did not discourage me. It gave me a clearer direction.

Since then, TDD has become much more than writing tests before production code. It has become the process through which I investigate a problem, discover behavior, make design decisions, and build confidence incrementally.

Two books were especially important in shaping how I understand this process:

• Test-Driven Development: By Example, by Kent Beck;
• TDD, by Jason Gorman.

The explanation I grew tired of repeating

Before adopting this process, I grew tired of repeatedly explaining the same thing to people who assigned me a task:

This change is simple. What takes time is manually testing the entire system to ensure that nothing has been broken.

The code change might require only a few lines.

The real cost came afterwards.

I needed to navigate through the application, recreate scenarios, remember which related features might be affected, and manually verify that an apparently unrelated behavior had not changed.

Then I needed to repeat much of that work after every adjustment.

Without automated tests, even a simple change carried a large and mostly invisible validation cost.

TDD did not merely help me write the change. It helped me build the mechanism through which I could safely change it again.

When the team does not want TDD

I respect teams that do not want to work with TDD.

I think it is unfortunate, but I accept it.

I introduce the idea once. I explain why I believe it would help. If the team decides not to adopt it, I do not turn the subject into a permanent argument.

Life goes on.

However, for my own implementations, I still prefer to work through tests.

When necessary, I create a private fork and maintain a branch containing the tests. I write the tests and implement the change there. Then I open the pull request containing only the production code expected by the project.

This is not my preferred outcome. I would rather have the tests maintained alongside the implementation so that the entire team could benefit from them.

But even when a project has insufficient test coverage, abandoning my process makes the work more difficult for me.

The private branch allows me to preserve the feedback loop I need to implement the change with confidence.

AI agents brought the anxiety back

AI agents changed my development workflow significantly.

They can explore a codebase, propose a plan, implement changes, execute tests, refactor code, and document decisions much faster than I could do all of those things alone.

But they also introduced new forms of anxiety.

I started asking myself:

Am I limiting the AI agent by requiring it to work through TDD?

How can I reproduce the TDD feedback loop when working with an AI agent?

Does the agent genuinely practise TDD, or does it develop the solution first and create the tests afterwards?

That final question bothers me the most.

An agent can produce a test and production code in the same implementation. The final diff may look exactly like the result of TDD.

But the final diff does not reveal the process.

Did the test fail before the implementation existed?

Did that failure provide useful information?

Did the test guide a design decision?

Was only enough code written to satisfy the next behavior?

Or did the agent first construct the complete solution and then create tests that confirmed what it had already built?

The resulting repository may contain the same files, but the development process is not the same.

The process still matters

In When Plan Mode Was No Longer Enough, I described how execution plans and smaller milestones helped me reduce the volume of code I needed to review after working with an agent.

That approach gave me more control over the size of each change.

But it did not reproduce the TDD experience.

A milestone can still contain production code, tests, and refactoring created together. It can be small enough to review while still hiding the sequence through which the design emerged.

This matters to me because TDD is not valuable only because of the tests left behind.

The sequence matters.

The feedback matters.

The small decisions made after each failure matter.

Most importantly, the process changes how I think while implementing the solution.

For years, TDD gave me a way to manage the anxiety of not knowing where to begin.

Now AI agents are forcing me to examine that process more carefully than ever.

I need to understand which parts of TDD must remain visible, which parts can be delegated, and how I can know whether an agent is following the process rather than merely reproducing its final appearance.

I will explore those questions in my next post.

Plan Mode initially felt like enough

My workflow with Codex CLI used to be relatively simple.

I would enter Plan Mode, discuss what I wanted to implement, refine the scope, question some decisions, and eventually reach a plan that looked reasonable.

Then I would leave Plan Mode and ask Codex to execute it.

For a while, this felt like a good separation of responsibilities. First, we would think. Then, the agent would implement.

But eventually, it stopped being enough.

My first disappointment with AI agents

The problem was not that Codex could not generate the code.

The problem was that it could generate too much code at once.

After a large implementation, I would receive a substantial set of changes that I still needed to understand, review, refactor, and validate. Sometimes the implementation was technically correct, but it contained decisions I would not have made myself.

Other times, the tests passed, but some of them were tied to implementation details instead of protecting actual behavior.

I started to feel exhausted after these sessions.

The agent had accelerated implementation, but it had also created a large review queue for me. I was no longer spending most of my energy writing code. I was spending it trying to absorb, validate, and reshape everything the agent had produced.

The bottleneck had moved.

Codex was not struggling to generate code. I was struggling to review that code responsibly.

This was my first real disappointment with AI agents.

Not because they were incapable, but because I realized that generating code faster does not automatically make the entire development process faster.

Shrinking the feedback loop

While searching for a better approach, I found the OpenAI Cookbook article Using PLANS.md for multi-hour problem solving.

It describes an execution plan, or ExecPlan, as a self-contained and continuously updated document that an agent can follow while implementing a complex change.

An ExecPlan is more than a list of tasks.

It includes the purpose of the change, the current context, the desired behavior, implementation milestones, validation commands, risks, decisions, progress, and lessons learned.

The concept that made the biggest difference for me was dividing the implementation into milestones.

Instead of asking Codex to execute the entire plan, I started asking it to implement one milestone at a time.

After each milestone, I could:

• review a smaller volume of code;
• refactor while the context was still manageable;
• verify whether the tests protected behavior;
• question design decisions;
• run focused validation;
• update the plan with what we had learned;
• decide whether the implementation was ready to continue.

Only after I considered the result good enough would I ask Codex to implement the next milestone.

The milestone became the unit of collaboration between me and the agent.

Turning the idea into a skill

The OpenAI Cookbook article is dated October 7, 2025, and its current version recommends gpt-5.2-codex.

After GPT-5.5 was released, I asked the model to revise the original instructions and make them more appropriate for the newer model.

The result required fewer step-by-step directions and placed more emphasis on the properties the final execution plan should preserve.

I then transformed those instructions into my implement-execplan skill.

The skill defines an ExecPlan as a living document. It requires the agent to maintain sections such as:

• purpose and scope;
• existing context;
• desired end state;
• implementation milestones;
• progress;
• decisions and their rationale;
• risks and mitigations;
• validation strategy;
• lessons learned.

It also requires each milestone to contain concrete changes, validation commands, and observable acceptance criteria.

However, the most important part of my workflow does not come only from the structure of the document.

It comes from controlling when the next milestone starts.

I do not want the agent to finish faster than I can understand the change.

A real example in seed4j-cli

A concrete example came from a feature I implemented in seed4j-cli.

The ChatGPT Android app allows me to use a GitHub integration that can read repositories. I used it to inspect the project and discuss the feature with GPT-5.5 in an experience similar to Codex CLI Plan Mode.

After refining the scope, I created the issue Enhance apply --plan with dependency chain status.

The goal was to improve the output of:

seed4j apply <module> --project-path . --plan

The command already displayed resolved parameters, but it did not explain whether the module dependencies had already been applied, were still pending, or required an explicit feature choice.

When it was time to implement the feature, I asked Codex CLI to read the issue and use my skill to create this execution plan:

2026-06-26_FEATURE_apply-plan-dependency-status-exec-plan.md

The work was divided into four milestones:

1. establish the living execution plan;
2. add the status of direct module dependencies;
3. add feature dependency satisfaction and pending choices;
4. update documentation and validate formatting.

This division gave me natural stopping points.

After the direct module dependency behavior was implemented, I could review it before introducing the more complex feature dependency rules.

After the behavior was stable, documentation and formatting could be handled separately.

What changed was not necessarily the quality of every generated line.

What changed was the size of the feedback loop.

The document became part of the work

There was another benefit I had not fully anticipated.

The execution plan became a record of the implementation.

During the work, Codex continuously updated the document with completed milestones, design decisions, validation results, discovered risks, unrelated repository problems, and lessons learned.

For example, the final document records why dependency planning remained in the CLI primary adapter, why CLI integration tests were used as the behavior contract, and why the dependency traversal needed to be deterministic and cycle-safe.

It also records that the repository-wide formatting check contained pre-existing failures unrelated to the feature, while the files changed by the implementation passed focused formatting validation.

Without the execution plan, much of that context would have disappeared inside the conversation.

The code would remain, but the reasoning behind several decisions would be lost.

Instead, I ended the implementation with both working code and a document describing how the implementation evolved.

In another post, I plan to explain what I started doing with these files instead of simply discarding them after the feature was completed.

What this approach still does not solve

Execution plans helped me control the size of generated changes, but they did not solve everything.

Codex can still create tests that are too closely tied to implementation details. A smaller milestone makes those tests easier to identify, but it does not automatically make them good.

This approach also does not reproduce the TDD experience I was used to.

In traditional TDD, the feedback loop begins with a failing test that expresses the next behavior. The design emerges through small cycles of failure, implementation, and refactoring.

An ExecPlan organizes implementation into milestones, but a milestone can still contain too much production code before the right behavioral tests are established.

I have made progress in this area by combining execution plans with other skills and more explicit testing workflows.

That deserves its own post because it changes not only how the agent implements a feature, but also how I participate during the implementation.

Regaining control without rejecting the agent

My first reaction to the review burden could have been to reduce my use of AI agents.

Instead, I changed the granularity of the work I delegated to them.

The answer was not a more detailed initial conversation followed by an even larger autonomous implementation.

The answer was a living plan, smaller milestones, explicit validation, and deliberate stopping points.

I still review the code.

I still refactor it.

I still question tests and design decisions.

The execution plan does not remove my responsibility from the process. It gives me a structure that makes that responsibility sustainable.

For me, that has become one of the most important lessons about working with coding agents:

The agent’s ability to generate code is not the only limit that matters.

My ability to understand, validate, and improve that code matters just as much.

From English prompts to Portuguese intention

I am Brazilian, and my native language is Portuguese.

Since the launch of ChatGPT, I got used to interacting with it in English. At the time, it was not just a preference. I clearly felt that the responses were better, and it also made my work easier when I wanted to generate code, documentation, method names, tests, and examples for my open-source projects.

Because most of the projects I contribute to use English as the standard language, using English with the AI felt natural.

But something changed.

Today, when I use Codex CLI for programming, I talk to it almost exclusively in Portuguese.

Even when the repository is in English.

Even when the documentation is in English.

Even when the generated code must follow English naming conventions.

And this still feels a little amazing to me.

Talking informally to the agent

The interesting part is not only the language itself.

The interesting part is how informal the interaction became.

When I am working on a feature, I usually talk to the AI agent in Portuguese, in a very natural way. I describe what I want, what I am worried about, what I think should be done first, and sometimes what I do not want it to do.

In many cases, before implementing the feature, the agent creates a Markdown document with the implementation steps.

That document is also written in Portuguese. 😀

Later, I want to share more details about how I usually get that document created, because this small step has become an important part of my workflow.

But even with that planning document in Portuguese, the implementation itself remains aligned with the project.

Method names are in English.

Tests are in English.

Feature documentation is in English.

The code follows the repository language, not the language I used to describe my intention.

The repository defines the language of the work

After looking more carefully at what makes this possible, my conclusion is simple:

I provide the intention in Portuguese.

Codex reads the repository.

It sees that AGENTS.md is entirely in English.

It understands that the project is standardized in English.

Then it applies my request using the language of the project.

That means the agent is not just translating my words directly into files. It is reading the surrounding context and adapting the output to the conventions that already exist there.

This is the part that I find fascinating.

The conversation can happen in one language, while the work product follows another language.

The seed4j-cli is the project I am working on, where I have experienced everything I mentioned 😀

Intention is not implementation

This made me think about the difference between intention and implementation.

When I talk to the agent in Portuguese, I am not asking it to make the project Portuguese.

I am using my native language to express intent with more precision, less friction, and more comfort.

The repository still has its own language.

The code still has its own conventions.

The documentation still has its own audience.

The agent becomes the bridge between those layers.

And maybe this is one of the reasons why working with AI agents feels different from only asking a chatbot to generate snippets. The agent is not only answering a question. It is operating inside a context.

Why this matters to me

For a long time, I associated “better AI results” with “using English.”

That still may be true in many situations.

But with an agent working inside a repository, the situation becomes more nuanced.

The quality of the result does not depend only on the language of the prompt. It also depends on the quality of the repository context, the project conventions, the instructions, the tests, and the way the agent is asked to reason before changing code.

This is a very different experience.

I can think in Portuguese.

I can explain the intention in Portuguese.

I can ask questions in Portuguese.

But the final result can still respect an English-based open-source project.

That feels like a small but important shift in how I work.

A question for other developers

I am still experimenting with this, but I am curious if other people are doing something similar.

Do you interact with an AI agent in a language other than English, even when the project itself uses English as its standard language?

And if you do, does the agent preserve the project language well?

I would love to hear how other developers are handling this.

Why I stopped writing about it

I had given up on writing about my progress with AI agents for a while.

Not because I stopped experimenting. Quite the opposite.

The problem is that everything changes too fast.

Every time I felt close to describing my workflow, I would improve something, replace a tool, change a prompt, adjust a habit, or discover a better way to interact with the agent. The idea of publishing a “complete workflow” started to feel wrong, because I knew it would become outdated almost immediately.

And I did not want to create an exhausting commitment for myself: writing a post today, then feeling forced to update it every time my process changed.

Why I did not want to sound prescriptive

There was another reason too.

I did not want to sound like I was trying to impose a workflow.

My biggest discoveries with AI agents have been very individual. They did not come from a perfectly synchronized team process. They came from trial and error, from frustration, from curiosity, from small improvements, and from many moments where I had to rethink how I work as a software developer.

So, for some time, I preferred not to write about it.

What changed my mind

But recently, after watching the podcasts from Kent Beck - Still Burning, I felt a strong desire to contribute more openly to this revolution that is happening.

AI agents are changing software development, but not only because of the tools themselves. They are also forcing us to rethink discipline, feedback loops, communication, design, testing, and the way we describe intent.

And I do not want to be only a consumer of this change.

I want to contribute to the conversation.

Maybe by sharing parts of what I am discovering, I can help someone else. And maybe, by making these ideas public, I can also accelerate my own learning through external insights, criticism, and suggestions.

A better format

So I started thinking about a better format.

Instead of trying to publish my entire workflow as if it were a recipe, I decided to share smaller steps from it.

• Pieces.
• Practices.
• Decisions.

Things that can be adapted, improved, questioned, or absorbed into someone else’s workflow.

That feels much more honest to me.

What I want to share

A complete AI agent workflow is too personal and too unstable to be treated as a final answer. But specific ideas can still be useful. A way to prepare context. A way to ask for changes. A way to review generated code. A way to avoid losing control. A way to keep tests and design as the center of the process instead of blindly trusting the agent.

That is what I want to share.

Not a definitive workflow.

Not a universal method.

Not “this is how everyone should work with AI agents.”

Just pieces of my current process, shared while they are still alive, still changing, and still being tested in real work.

I will start doing that soon.

It’s been a little over a year since I realized that I work in the desert (a term coined by Kent Beck), and I understood why some things that are so useful and cool in the forest didn’t work when I tried to bring them into the desert.

Talking to people outside my reality

So I decided to talk to people who are outside my reality to see how their day to day works, what new pain points they have that I’ll never experience in the desert because I’m dealing with more “basic” problems.

Learning from open-source friends

For a while now, I’ve been talking to people I work with on open-source projects (e.g., seed4j.com), and even though the project is incredible, it’s impossible to apply it in the desert where I work. I was fascinated to learn how they use it in their daily routine. It’s not just exciting because it’s something sophisticated; it’s genuinely useful and makes the day to day less stressful and overwhelming.

Replying to a newsletter

Recently, I received an email from a newsletter (I’ve subscribed to several), and what caught my attention was the frankness of the writing and a genuine invitation to reply because the author was interested in talking and getting insights from other people’s perspectives. I replied, and I’ve kept talking with him, someone who, in my opinion, works in the forest. These are interesting conversations that help me discover new things that my reality would hardly ever provide.

Catching up with a developer friend

On top of that, I had the chance to talk over a video call with a developer friend I hadn’t spoken to in a long time. I was impressed to discover new areas where a developer can work, areas I didn’t even know existed. In the end, it felt like a conversation with a longtime friend. It had been a long time since I’d spoken English, but I felt so comfortable that at no point did I feel that speaking English was getting in the way.

Back to Seed4J CLI

I decided that I’m going to get back to working on the Seed4J CLI, but before that, I decided to reach out to other developers who use Seed4J to understand how they use it day to day and to learn a bit about their work routine. Surprisingly, a developer has already replied and supported me, saying that creating this CLI application might be a good path so that the AI can autonomously understand which features Seed4J has and use them whenever appropriate, making the experience more transparent.

Motivation boost

This contact with other people in the software development industry, people living realities different from mine, gave me an extra boost of motivation. And the fact that everyone treated me with so much kindness and attention made me feel good.