More software developers each day are using AI software development agents (e.g. Open AI Codex, Claude Code, etc.) to build, maintain, and debug apps. Anecdotally the process is called vibe coding, and there are a couple ways people tend to work:
The informal chat workflow: Simply start chatting with the AI, describing what you want, correcting or confirming responses, reviewing the built application, and repeat until happy with the result. This is the easiest process to use, and for smaller less complex apps it can work. But it takes much more time in the end, and the resulting app likely has security and/or performance holes, among other issues.
The spec-driven workflow: This process requires more planning and is usually where serious vibe coders end up. This process involves writing a specification for your app (as one or more markdown files), using an AGENTS or README file to document where and what the spec is, more or less, and then proceed with workflow #1.
Although AI agents can be amazing at inference and accuracy, the gremlin in the machine is the fact that AI models don't think; they simply return probabilistic token predictions. That's why the same prompt can result in different responses, some of which may conflict with prior responses. For example, ask an AI what the most popular oil is, and you may get "olive oil" as the result most of the time, but then out of the blue you may get "Mobile 1". Specifying "cooking oil" would certainly help. But you may still get something other than "olive oil" every so often.
Besides AI models being probabilistic I would argue that perhaps the most important flaw is the fact that human language is too imprecise to be used for any meaningful software development. This is why the same series of chat messages will produce a different result, as if two different people were asked to create the same app independently.
This can also yield interesting security flaws. For example, I have seen vibe coded apps that appeared feature complete, passed the AI generated unit tests, and yet had "hidden" features woven into the code that were never requested. This can allow unauthorized access, crash opportunities for incomplete feature code, and a myriad of other exploitation vectors. This behavior is by design, but inherently insidious for the human at the wheel.
The imprecision of human language is also why markdown specs don't guarantee reliable, performant software development. It's just too difficult to write a spec with enough detail and nuance to ensure the AI agent builds exactly what you want. You'll never answer every important question, and you'll always forget important details. This is also why the waterfall development process was largely abandoned.
Software development shops and enterprise dev teams have been relying to some degree on code reviews to identify and remediate bad AI generated code. Some of these workflows even require a sign off review in order to merge the code into a production branch for deployment. The thought is that having experienced developers checking things will fill that gap.
There are two issues with this approach.
If your AI generates 10,000 lines of code in a single day, it could take a week for a human developer to meaningfully review and understand that code. At that pace AI becomes the bottleneck.
Software developers don't want to review code all day. They want to understand and solve problems by building things.
This results in an industry that thinks it filled the gap with code review even though not much actual code is being reviewed.
One workflow that may become the de facto standard for software development is a variation of test-driven development (TDD) called acceptance test-driven development (ATDD). How does it work?
Acceptance test-driven development is a way of building software where you first define what the finished software should do and how you will know it works before asking an AI agent to write the code. These expectations are written as clear, real-world examples of how someone should be able to use the software and what should happen in response. But instead of writing these scenarios as prose, they are written as tests. As you add tests for specific features, the AI agent sees that they're failing, and implements the code to make the test pass. You end up with a complete software application that is also testable! In fact, you could give another AI agent the same tests and it will also build a fully functioning app, even if the code varies a bit.
Instead of simply trusting that the AI has produced something that looks right, you have an objective way to verify that it does what was actually requested. This creates a shared understanding between you and the AI, keeps development focused on the desired outcome, and makes it easier to catch mistakes before they become bigger problems. The tests ARE the spec!
The workflow is straightforward, albeit deep in flexibility. Here's one way to use it:
First, set up an empty project, and two test projects, one for ATDD tests, the other for AI-generated unit tests (yes, unit tests are also used to ensure reliability).
You can add an AGENTS file to help orient the AI agent regarding what you're building and how you want to work. Here's an example you can add to your AGENTS.md file:
Next, in the ATDD test project start writing tests against a domain-specific language (e.g. pet store) for a specific scenario.
This code will not even compile. You made it all up.
Lastly, you then tell the AI agent to get it passing by using ATDD and implementing two things:
A domain-specific language (DSL) which will use a protocol driver to perform actual work against the application.
A protocol driver which is interface code that allows the DSL to call the application to get work done.
Fancy names, but basically the DSL houses the made up test methods, like GivenProductInStockAsync() which call the driver code to perform the work. The AI agent will understand this workflow. It will write the code for both of those layers, and also write the code in the application that ultimately gets tested. This pattern uses a Given/When/Then (GWT) naming convention/structure that makes it easier to read scenarios as domain-specific english language.
Here's a sample prompt template for the scenario test cycle:
You then remediate and repeat until the test scenario works the way you want at the current stage of development.
With enough tests and good coverage, an app will be built; a testable app. Early in development features that pass won't perform all the correct actions. For example, adding a login may work, but it may not use an API endpoint. Later tests may implicitly make that a requirement in which case the earlier login code will get modified by the agent in order to keep the newer tests passing.
And the best part of ATDD for a software developer is that the act of writing and testing these scenarios still satisfies their need to solve problems and build things; secure, performant, testable things.
There's usually more to the story so if you have questions or comments about this post let us know!
Do you need a new software development partner for an upcoming project? We would love to work with you! From websites and mobile apps to cloud services and custom software, we can help!
