Architecting with LLMs: use plain old code when you can, LLMs when you must

I recently wrote about the Coding agent sandwich, which is a design pattern for a coding agent harness. The pattern consists of a tasty filling of LLM invocations sandwiched between two slices of plain old code (POC). Its practical rule is: use POC when you can, LLMs when you must.

The following diagram shows how the pattern’s worked example, the implement plan workflow, applies this rule. The workflow is decomposed into a tree of responsibilities. Each responsibility is implemented using either POC (gray) or an LLM (yellow). LLMs are used only for the responsibilities that require judgment, such as writing code and fixing a broken build. The workflow has orchestration at two levels. POC orchestrates the overall workflow, and the LLM agent that implements each task decides which action to take next within that task.…