Each testing level plays a specific role in software development and answers a distinct question:
Unit Tests - “Has a function changed that was supposed to remain unchanged?” They test the smallest isolated pieces of code (individual functions or methods) independently from the rest of the system. (A detailed analysis of unit tests is covered in a separate blog post.)
Integration Tests - “Do components still work together correctly after a change?” They test real connections between modules (e.g., database + service layer + API), rather than just isolated functions. They uncover scenarios where individual parts pass on their own but fail when combined.
E2E (End-to-End) Tests - “Does the system work as a real user would experience it?” They simulate complete user workflows (login → action → result) through the actual UI or API. They catch regressions that neither unit nor integration tests can detect because these issues only surface within a real workflow.
(There are, of course, other types of testing—such as *Smoke tests**, load testing, or security testing—which are outside the scope of this article.)*
All automated testing levels share a common goal: to verify that functionality hasn't broken outside the scope of the task currently being worked on. A task might be completed perfectly according to specifications, yet still break something else as an unintended side effect. Without a testing layer that extends beyond the specific change, no one would notice until a user reports the issue in production.
The Essential Role of Manual Testing
Despite advanced automation and AI integration, manual testing maintains its indispensable and primary role. A human tester brings real judgment, intuition, and the ability to detect subtle nuances or unexpected UI behaviors that automated scripts easily miss. Manual verification was, is, and always will be what gives the development team 100% confidence that the application is ready for real users.
AI-Assisted Development and Compensating for "Blindness"
In the era of AI-assisted development, an automated testing layer is even more critical. An AI agent cannot perceive the entire system at once, making side effects more likely than with a human developer who carries the full context and history of past decisions. Tests serve as a mechanism to systematically compensate for this system blindness before code reaches final human approval.
Practical Impact: The Challenges of E2E Testing with AI
Creating E2E tests is particularly complex for AI. It is not just about writing the test script itself, but primarily about preparing suitable test data so that the software can use it (often for just a small part of the test). Preparing this data is highly demanding.
Consider a seemingly simple action: clicking a dropdown list and selecting a parameter (e.g., document type Invoice). If the test fails or does not complete, it is crucial to diagnose the exact root cause. A variety of situations could have occurred:
- The AI lost its way and couldn't find the dropdown element in the UI (even though it's present).
- The test hit an earlier error along the path and couldn't recover.
- The dropdown exists, but does not contain the expected data for selection.
- The AI resorted to using JavaScript (an invisible workaround), successfully setting the value programmatically while leaving it invisible to an actual user.
Numerous issues can arise around a single dropdown. The key is whether the AI (or evaluation system) can distinguish a genuine bug from a cascading failure caused by something else. Simply saving data successfully does not guarantee everything is working properly.
Behavioral Approach and Test Coverage
For this reason, tests should always focus on the behavioral aspect of the system, testing what we actually expect to happen:
Happy Path (~90% of cases): Primary focus is given to what users will do the vast majority of the time.
Critical Edge Cases: Limit conditions that don't occur frequently, but are vital to system integrity.
Other less probable combinations are not pre-tested. If they occur in production, they are fixed on the fly or prevented via software constraints and validations (e.g., disabling navigation until inputs are valid).
E2E Testing in AITM
Within AITM, E2E testing represents an explicit and mandatory pipeline phase for tasks impacting the UI (determined by task type/intent). It is not an optional addon to skip when deadlines approach—a pipeline cannot be marked complete without it.
If an E2E test is missing or the environment is unavailable, the build fails by default (with rare exceptions where a failed E2E is permitted to proceed). Additionally, every E2E execution captures screenshots that are visible in the task results.
How E2E Works Currently:
- Before running tests, the build is guaranteed to be successful and executes in its own isolated worktree.
- Database state is prepared with specific test data alongside default records.
- Testing focuses strictly on the Happy Path (does not cover all possible scenarios).
Future Roadmap for E2E:
- Flow Maps - The system constructs a path map to given elements linked directly to business logic (already integrated in AITM).
- Autonomic Navigation - Tests read the live path map, knowing precisely where to click, type, and verify (currently undergoing testing).
- State Comparison (Visual Diffing) - Every step is recorded with screenshots taken before and after code modifications to confirm that expected elements and data match DB or UI expectations.
- Business Logic Verification - Automated checking of expected output data (e.g., verified invoice totals—currently in development).
- Nightly Integration Runs - End-to-end integration tests will be generated automatically from step states and software requirements to run overnight. Any failures will generate tasks or request human verification (as AI can make mistakes and needs human sign-off).
- Advanced Scenarios - Future releases will incorporate invalid parameter injection, multi-instance concurrency testing, and edge case simulations.
Summary of E2E Core Principles
The essence of effective E2E testing relies on managing maps and data entity relationships. For 90% of software, it boils down to the standard lifecycle:
#AITM #Data Input Validation #E2E #Integration test #AI TESTING