← Back to Blog

When AI starts writing software that is no longer simple

When AI starts writing software that is no longer simple

Today, AI-assisted software development often follows a straightforward process:

  1. Write a task description (prompt).

  2. Let AI produce an analysis and proposed solution.

  3. Prepare skills, agents, and subagents.

  4. Start the implementation.

  5. Let AI write the code, test it, and fix any detected issues.

At first glance, this looks like an ideal development workflow. AI operates autonomously, distributes tasks among agents, writes tests, and gradually completes the requested functionality.

And for smaller projects, this approach can work extremely well.

If you are building a simple e-commerce store, a five-page company website, or a basic administration system, AI can often handle a significant portion of the work with very good results.

Why?

Because millions of developers have already built similar software.

Imagine two online stores. One sells toys, while the other sells apples.

From a customer's perspective, these are two different businesses. From a software architecture perspective, however, they operate in very similar ways:

  • Products organized into categories.

  • Pricing and inventory management.

  • Shopping carts.

  • Order processing.

  • Payment methods.

  • Customer management.

What changes is primarily the data, specific business rules and certain features. The core architectural principles remain largely the same.

AI recognizes these patterns because it encountered similar structures during training.

The problem begins when you stop building a conventional application and start developing large-scale enterprise software.

What happens when a system contains 300,000 to 500,000 lines of code?

Imagine an enterprise application containing hundreds of thousands of lines of code, dozens of modules, hundreds of dependencies and complex business rules.

Such a system may have evolved over several years. During that time, architectural decisions, exceptions, historical dependencies and business rules have accumulated.

Not all of them are documented.

A single method may affect ten other modules. Changing one parameter may disrupt invoicing, user permissions or order processing.

Now imagine introducing an AI agent into this environment.

You give it a task:

"Modify the payment process to support an additional payment method. Preserve backward compatibility and test everything."

The agent creates a plan, distributes work among subagents, implements changes, runs tests and returns the results.

Everything may appear correct.

But passing tests does not necessarily mean that the software has preserved all existing business rules.

And this brings us to the problem that completely changed my approach to AI agents six months ago.

The experiment that changed my view of agentic systems

Six months ago, I conducted an experiment that led me to take a different path instead of simply building more agents and subagents.

Until then, I had operated under an assumption shared by many developers:

The better I distribute tasks among specialized agents and the more validation mechanisms I introduce, the more reliable the result will become.

But during my experiments, I encountered a fundamental question:

How can an agent correctly implement a business rule if it does not even know that the rule exists?

And how can another agent reliably verify the result if it also lacks complete information about how the system is supposed to behave?

I began examining how context is transferred between agents.

What I discovered was that the way information is passed along and interpreted can become one of the weakest points in the entire process.

How do agents and subagents actually work?

In practice, there are two common approaches to communication between AI agents.

1. Hierarchical Model – A main agent coordinates subagents

This architecture is commonly referred to as Hub-and-Spoke or Star Topology.

The principle is straightforward:

  1. The user submits a task to the main agent.

  2. The main agent analyzes the task and breaks it into smaller parts.

  3. It creates or activates subagent A and provides instructions.

  4. Subagent A performs the work and returns a result.

  5. The main agent evaluates the result and may delegate further work to subagent B.

  6. Once all parts are completed, the main agent combines the results.

Subagent A does not necessarily know what subagent B is doing. Communication and information exchange are coordinated through the main agent.

Advantage: Centralized control, easier state tracking, clearer responsibility and a lower risk of uncontrolled communication between agents.

Disadvantage: The main agent becomes the central point through which task instructions and results are interpreted. If it omits important information while delegating work, a subagent may operate with incomplete context.

2. Peer-to-Peer – Agents communicate with each other

The second approach involves direct or continuous communication between agents, for example through a shared conversation thread or a messaging system (*Message Bus*).

Imagine the following scenario:

  • Agent A writes code and publishes a CodeWritten event.

  • Agent B, responsible for testing, receives the event and starts the tests.

  • Agent C simultaneously prepares documentation.

  • Agents continuously exchange results and react to changes.

Advantage: Parallel task execution, fast information exchange, and reduced dependency on a single central agent.

Disadvantage: More complex coordination, the risk of working with outdated information, conflicting modifications, and greater difficulty tracing the root cause of errors.

Both approaches can be extremely useful.

But neither solves the fundamental problem on its own.

An agent may have a perfectly defined task, but without understanding the system's actual business logic, it can produce a technically correct solution that is wrong from a business perspective.

Where do errors arise in agentic workflows?

AI is not a deterministic compiler that translates instructions directly into source code. A language model interprets available information and generates solutions based on that context.

When information is missing, it may make incorrect assumptions. When the available information is incomplete or contradictory, it may choose an inappropriate approach.

Let's examine a typical workflow for a larger task.

Step 1 - Interpreting the user prompt

The main agent receives the task and attempts to understand the requested change.

At this stage, it may overlook a dependency, misunderstand a constraint or assume something that was never explicitly stated.

Step 2 - Creating instructions for a subagent

The main agent creates a more specific task description. It selects information from the original prompt that it considers relevant.

However, if it does not know certain business rules, it cannot pass them on correctly.

Step 3 - Implementation

The subagent receives its instructions, analyzes part of the codebase, and selects an implementation strategy.

It may produce syntactically correct, high-quality code while unintentionally changing behavior that should have remained unchanged.

Step 4 - Evaluating the result

The main agent receives the subagent's output, evaluates it and decides whether the task is complete.

But it may still be operating under the same incomplete assumptions as before.

Step 5 - Testing and Fixing Errors

Unit tests, integration tests, E2E tests, static analysis and documentation reviews follow.

These mechanisms significantly improve software reliability, but they have limitations.

A test verifies specific expected behavior. If an important business rule is missing from the test specification, the implementation may pass every test while violating that rule.

And when AI fixes one issue, it may unintentionally affect another part of the system.

This creates a familiar cycle:

Implementation → Testing → Fix → Regression → More Testing → Another Fix.

Therefore, simply increasing the number of agents or repeating tests more frequently does not automatically guarantee a correct result.

Why can errors accumulate across an agent chain?

Let's consider a simplified mathematical model.

Assume we have three consecutive steps, each with a hypothetical 95% probability of being executed correctly.

If these steps are independent and all three must succeed, the probability becomes:

0.95 × 0.95 × 0.95 = 0.8574

That represents approximately an 85.7% probability that all three steps will succeed.

With ten such independent steps, the result would be:

0.95¹⁰ ≈ 59.9%

This is an illustrative calculation, not a measured success rate for AI agents. In real systems, errors are not always independent, validation mechanisms can catch mistakes, and individual steps can correct previous failures.

However, the example demonstrates an important principle:

The reliability of individual agents does not automatically guarantee the reliability of the entire chain.

When incomplete instructions are repeatedly interpreted at each stage, additional opportunities for incorrect decisions can emerge.

That is why I believe simply adding more agents, tests, and validation mechanisms is not enough.

We need to change how AI receives information about the system itself.

The key difference: Tests verify results, Business Logic defines boundaries

The conventional approach to AI-assisted development is reactive.

You submit a requirement, AI implements a solution and tests are then used to determine whether something has been broken.

That is certainly better than having no tests at all.

But there is still a fundamental weakness:

Tests cannot reliably validate business rules that nobody knows about and that are not part of the testing specification.

That is why I believe a better approach is to provide AI with information about the system's actual behavior before implementation begins.

Not only what needs to change, but also:

  • Which business rules apply to the affected code.

  • Which other modules depend on the functionality.

  • Which inputs and outputs must remain compatible.

  • Which scenarios must remain unchanged.

  • Which exceptions and historical constraints exist.

  • Which parts of the implementation AI may modify and which it must preserve.

This significantly reduces the amount of behavior AI must infer on its own.

Two Approaches to the Same Task

Approach A - Prompt + Testing

AI receives a task, chooses an implementation based on the available context, produces the solution and runs tests.

If the tests are incomplete, violations of existing business rules may go undetected.

Approach B - Prompt + Business Logic + Testing

AI receives the task together with relevant rules, dependencies, contracts, and constraints.

As a result, it knows which behavior must be preserved before implementation begins.

Tests then do more than detect errors. They also verify specific business contracts.

The difference is not that we stop testing.

The difference is that we try to prevent errors before they are introduced.

When AI starts writing software that is no longer simple — screenshot 1

Practical Example: Adding a new payment method

Imagine that we have a file called CheckoutService.ts, which handles payments in an e-commerce application.

We want to add card payments as an additional payment method.

Option A: A conventional prompt

Task:

> Add card payments to CheckoutService.ts. Preserve existing functionality and run all tests.

AI modifies the code, adds a new payment branch, and updates the tests.

All tests pass.

The task is marked as complete.

However, the system contains an important business rule:

Company accounts must use bank transfer as their default payment method.

This rule was introduced several years ago. The developer who implemented it no longer works at the company. The rule is not clearly documented, and no dedicated regression test exists.

AI introduces card payments as the default method for all customers.

From an implementation perspective, the new functionality works correctly.

From a business logic perspective, the system is broken.

And the tests failed to detect the problem.

Option B: A structured task with Business Logic in AITM

Now imagine that AI receives the same request, but this time the task includes the relevant business contract, related variables, links to historical tasks, and precise boundaries for permitted changes.

The task could look like this:

READ CLAUDE.md first.

TASK:

Add card payment as an additional payment method

in CheckoutService.ts.

Contract:

%CHECKOUT-019 — "For company accounts
(companyId != null), bank transfer must remain
the default payment method. Card payment may
be offered only as an additional option."

Related variables:

BL.CHECKOUT.PAYMENTMETHOD_DEFAULT
BL.CHECKOUT.PAYMENTMETHOD_OPTIONS

Related task:

#<link to the task that originally implemented bank-transfer-as-default checkout>

In scope:

- Add a new "card" option to the available payment methods in CheckoutService.ts.

- Expose it as a selectable option in the existing payment method selection flow.

Out of scope:

- Changing the default payment method for any account type.

- Any UI/checkout-flow redesign beyond adding the new option.

- New payment provider integration logic beyond the method registration itself.

Must not break:

- Existing default payment selection logic (company accounts → bank transfer stays default).

- Company account checkout behavior end-to-end.

- Existing bank transfer processing path.

- Compatibility with the current payment API contract.

File path:

src/Services/CheckoutService.ts
Confirm the exact path via contractsForFile before editing.

Verify:

- Company account checkout still selects bank transfer by default
  (companyId != null).

- Card payment appears as a selectable, non-default option.

- Existing bank transfer processing is unchanged.

- Run unit and integration tests covering CheckoutService.

More information on how this works in AITM: https://aitm.cz/Docs/KnowledgeSharing

Why is this format important?

At first glance, it may look like nothing more than a more detailed prompt.

But the real difference is that the task is connected to information that can be retrieved and verified against the actual project.

1. Contract %CHECKOUT-019

The identifier %CHECKOUT-019 is not just a textual note.

It is an illustrative contract identifier that the agent can retrieve through GraphQL operations such as contractsForFile or contractsForVariable.

Instead of relying solely on the rule written in the prompt, the agent can retrieve its structured representation and verify it against the current codebase.

2. Business Logic Variables _BL.

Identifiers such as _BL.CHECKOUT.PAYMENT_METHOD_DEFAULT and _BL.CHECKOUT.PAYMENT_METHOD_OPTIONS represent business logic expressed through AITM's communication language.

They help connect a specific requirement to the rules and variables that determine system behavior.

3. Reference to a related task

The Related task section references the original task in which the business rule was implemented.

This gives AI additional context, not only about how the system currently behaves but also about why a particular decision may have been made.

4. Clearly defined change boundaries

The In scope, Out of scope and Must not break sections define three separate categories:

  • What AI must implement.

  • What AI must not implement.

  • Which existing behavior AI must preserve.

Instead of receiving an open-ended task that it can interpret or expand freely, AI receives a precisely defined implementation scope.

5. Verification before writing code

The file path is not automatically assumed to be correct.

The agent is instructed to use contractsForFile first, confirm the actual path, and identify the associated contracts.

Only then should it begin modifying the code.

6. Prompt validation through TPV

An important part of the AITM approach is the TPV prompt validator.

It checks mandatory task sections before the task is submitted for execution.

For example, it verifies the presence of scope definitions, restrictions, contracts, file paths, and verification requirements.

This moves part of error prevention from testing completed code into preparing the task itself.

Note: The contract identifiers and business logic variables in this example are illustrative placeholders. In a real project, the optimizer retrieves them through GraphQL from the specific project. They should not be guessed or generated without verification.

What changes with this approach?

In the conventional approach, the agent first interprets an open-ended task and decides for itself which files, dependencies, and business rules matter.

With AITM, the task is connected to specific contracts, business logic variables, historical tasks and clearly defined change boundaries.

The agent can therefore make decisions based on verifiable information rather than relying primarily on assumptions.

And that is where I see the fundamental difference.

Business logic is no longer merely part of the documentation. It becomes part of the implementation control process itself.

This is the principle behind AITM

When we started designing AITM (AI Task Manager), we did not want to build just another system that launches agents, distributes tasks, and checks their results afterward.

We focused on a different question:

What if AI could receive relevant business logic directly from the existing source code before making any changes?

Not just documentation someone wrote a year ago.

Not just general instructions stored in a skill.

And not just a few files that an agent happens to select.

Instead, structured information describing how the affected part of the system actually behaves, which contracts govern it, and what other components depend on it.

The core principle of AITM is therefore to enrich the working prompt with business logic identified from the relevant parts of the codebase.

The agent receives not only information about what it must do, but also information about what it must preserve.

This is particularly important in large-scale software systems, where a single prompt may not contain every relevant dependency or constraint.

Business Logic as guardrails for AI

Think of business logic as guardrails along a road.

Without guardrails, AI can take many different paths. Some may lead to a correct solution, while others may violate hidden rules within the system.

By providing relevant contracts and constraints, we significantly narrow the space for undesirable decisions.

AI can still independently choose implementation strategies, algorithms and code structures.

But it must respect the known business rules.

Subsequent testing then verifies whether those rules were actually preserved.

This represents a fundamental change in philosophy.

We are not simply trying to fix errors that AI creates. We are trying to reduce the probability that those errors are introduced in the first place.

Does this mean we no longer need agents and tests?

Absolutely not.

Agents are useful for parallel execution, specialized activities and automating large tasks.

Unit tests, integration tests and E2E tests remain essential components of software quality assurance.

But agent orchestration and automated testing cannot replace knowledge of business logic.

If an agent does not understand what the system must preserve, even a perfectly organized team of subagents will not solve the underlying problem.

And if tests do not cover the relevant scenarios, every test may pass while the application still behaves incorrectly from a business perspective.

The right approach is not to choose between business logic and testing. It is to combine them.

First, identify the system's actual behavior.

Then express that behavior as clear rules and contracts.

Next, provide these rules to AI together with the task.

Only then proceed with implementation, validation, and testing.

This approach can reduce unnecessary fixes, regressions, and repeated implementation cycles.

Conclusion: it is not about the number of agents. It is about the quality of information.

For simple applications, AI can deliver excellent results even with relatively short instructions.

But with enterprise software containing half a million lines of code, it is not enough to assume that an agent will understand everything correctly, discover every dependency and then flawlessly test its own work.

The larger the system becomes, the more important it is to understand and protect its actual business behavior.

You can have ten agents collaborating perfectly. But if none of them knows a critical business rule, they can collectively produce a perfectly tested solution that is fundamentally wrong.

That is why we take a different approach with AITM.

First, context, rules, and contracts.

Then, implementation.

And finally, validation and testing.

Because the goal is not for AI to write as much code as possible in the shortest amount of time.

The goal is for AI to change exactly what needs to be changed, without breaking what already works.

www.aitm.cz