The shortest feedback loop in software development is not between developer and test suite. It is between the person writing a feature and the person who will use it. Everything else — unit tests, integration tests, code review, CI pipelines — exists to approximate that loop when direct contact is not possible or not fast enough.

The two-question invariant

When a test engineer asks „how should I test this feature,“ the question is structurally identical to „what is the core functionality and why does it matter to the customer?“ These are not two questions — they are the same question expressed from two positions in the process. A test that cannot be derived from a clear answer to the second question is not a useful test. A feature description that cannot be turned into a test is not a usable specification.

This is not a testing observation. It is a communication observation. The function of the test design review — in any methodology — is to expose misalignments in the shared mental model of the system before those misalignments are encoded into code or test data. When a test engineer cannot formulate a test for a requirement, that failure of formulation is a signal: the requirement contains a hidden assumption, an ambiguity, or a contradiction. The test design process is a structured protocol for extracting these hidden properties from natural-language descriptions.

Errors are easy. Conceptual problems are not.

An error is a localised deviation: a wrong calculation, a missing null check, an off-by-one. Static analysis, automated tests, and linters catch errors reliably. Their detection cost has been declining for decades and continues to decline. LLM-assisted code generation accelerates this further — the syntactic quality of generated code is generally high, and common error patterns are well-represented in training data.

A conceptual problem is a deviation in the model of the system: a feature that correctly implements a wrong interpretation of a requirement; a data structure that accurately represents a domain concept that turns out to be the wrong concept; an integration pattern that coherently connects two services according to a contract that was never actually agreed. Static analysis cannot detect these. Automated tests cannot detect them unless the tests were written against a correct specification — and if the specification was wrong, the tests are wrong too, and they will pass.

This asymmetry — errors are findable, conceptual problems are not — is the persistent structural challenge of software quality. It does not diminish with tooling improvement. It intensifies. Faster delivery cycles reduce the time available for the communication that would surface conceptual problems before they are encoded. AI-assisted coding tools increase the throughput of code generation without increasing the precision of requirement understanding. The ratio of conceptual-problem risk to error risk increases.

AI as amplifier — not as specification validator

LLM-based coding and documentation tools are, at their current capability level, high-quality amplifiers of what is already in the prompt context. They are excellent at generating syntactically correct, stylistically appropriate code that matches the intent expressed in the prompt. They are poor validators of whether the intent expressed in the prompt matches the intent held in the customer’s head.

This is not a criticism — it is a description of the tool’s operating range. A signal amplifier amplifies signal and noise equally. If the input to an LLM coding tool contains a conceptually wrong specification, the output will be a larger, more refined, better-formatted version of the conceptually wrong specification. The error detection rate for implementation errors will be high. The error detection rate for conceptual errors will be low or zero. The test engineer’s role in this environment is not to compete with the tool on error detection. It is to maintain the discipline of asking „does this specification correctly reflect what the customer needs?“ — the question the tool cannot answer.

Short feedback loops as quality infrastructure

The practical implication for team communication: the feedback loop between „what are we building and why“ and „how will we verify it“ must be kept shorter than the loop between „writing code“ and „deploying it.“ In a sprint-based team using AI coding tools, this means the test analysis meeting (or equivalent exchange) happens at story definition, not after code completion. The developer, the test engineer, and the product owner are in the same room (physical or virtual) when the acceptance criterion is written, because that is the moment when the conceptual model can still be corrected at minimum cost.

The argument is not philosophical. It is economic. The cost of correcting a conceptual problem discovered during acceptance testing is an order of magnitude higher than the cost of correcting it during story definition. In a team generating high volumes of code via LLM tools, the volume of code produced per unit of communication investment increases — which means the cost-per-defect of conceptual problems increases proportionally. The communication investment does not scale with code volume. The testing investment must.

References

  • Deming, W.E. — Out of the Crisis (1982). MIT Press.
  • DeMarco, T., Lister, T. — Peopleware: Productive Projects and Teams, 3rd ed. (2013). Addison-Wesley.
  • Beck, K. — Extreme Programming Explained: Embrace Change, 2nd ed. (2004). Addison-Wesley.
  • Weinberg, G.M. — The Psychology of Computer Programming (1971 / 1998 Silver Anniversary ed.). Dorset House.
  • Wiegers, K., Beatty, J. — Software Requirements, 3rd ed. (2013). Microsoft Press.
  • Feathers, M. — Working Effectively with Legacy Code (2004). Prentice Hall.