# AI coding makes dependable delivery the next engineering challenge > When AI increases code output, engineering managers need to see where changes wait, what validation costs and whether released software behaves as intended. By BIG CHANGE Editorial Published: 2026-09-22T05:31:05.772Z Updated: 2026-09-22T05:31:05.772Z Canonical: https://bigchange.ai/blog/ai-coding-ci-bottleneck-dependable-delivery ![A large stack of blank work cards waits on a conveyor before a magnifying-glass inspection station, with a smaller stack in the output tray.](https://bigchange.ai/api/media/file/validation-queue-hero-v1.png) AI-generated conceptual illustration by BIG CHANGE. More proposed changes can put pressure on validation. Conceptual illustration of a queue and inspection stage; stack sizes are not measurements, and passing checks does not guarantee dependable delivery. An AI coding agent can finish a patch while the checks on its previous patch are still running. For an engineering manager, that creates an awkward distinction: the team has produced more code, but the customer has received nothing yet. Linear offers a useful case. In a September 21 engineering account, Mufeez Amjad reports that its test suites almost quadrupled from the start of 2026 while pull-request waits on continuous integration fell from more than six minutes to just over five. Runner time per test roughly halved. Those are separate improvements; the last figure does not mean the total CI bill halved. Linear attributes growing pressure on validation to agent-assisted development, but its account is not a controlled experiment establishing AI's productivity effect. [Linear's engineering account](https://linear.app/now/ci-bottleneck-reworked). Continuous integration, or CI, runs automated checks on proposed software changes. It sits between a plausible patch and a change a team is prepared to accept. When AI coding increases the supply of patches, that checking capacity becomes a business concern: how much useful work can the organization safely absorb? Our view is that managers should make three outcomes visible together: time to trustworthy feedback, total computing work and the reliability of released changes. A faster answer is valuable only if the team understands what that answer establishes. ## A shorter wait can involve more work Imagine a test suite containing twelve minutes of work on one machine. Splitting it evenly across four machines could finish that work in three minutes. This hypothetical assumes perfect balancing and ignores everything else. The machines still perform twelve minutes of work in total. Now give each machine one minute of preparation. The four-machine version takes four minutes to finish and consumes sixteen runner-minutes. The single-machine version takes thirteen minutes and consumes thirteen runner-minutes. Waiting time improves sharply while aggregate machine consumption rises. Actual charges also depend on runner types, pricing and billing rules. This distinction matters when a dashboard celebrates a shorter green bar. A manager buying parallel capacity is purchasing earlier feedback, potentially at an additional cost. That can be an excellent trade if it removes a genuine constraint. It is harder to justify when the completed change then waits hours for an available reviewer. The unit of analysis should extend beyond one CI run. A patch that triggers repeated revisions, reruns and a second round of review may consume considerably more work than its fastest individual run suggests. Count the attempts needed to accept a change, including abandoned attempts. Otherwise an efficient-looking pipeline can conceal an expensive development process. Nor is every waiting minute an equivalent labor cost. A developer who starts another useful task while checks run has not necessarily lost that whole interval. Repeated interruptions and returning to unfinished work can still hurt. Measuring those effects requires observing how the team works, rather than multiplying elapsed minutes by salaries. ## The research supports a mixed picture DORA's March 2026 analysis reports two distinct bodies of evidence. Its 2025 research associated higher AI adoption with both greater software-delivery throughput and greater instability. Separately, researchers analyzed 1,110 open-ended responses from Google engineers collected in the third quarter of 2025. Engineers described help with drafting and other tasks alongside verification overhead. The internal qualitative study is not the population behind the broader DORA association, and neither establishes that AI caused a particular team's release problems. [DORA's analysis and methodology](https://dora.dev/insights/balancing-ai-tensions/). There is also stronger experimental evidence with a narrower scope. METR's July 2025 randomized study assigned 246 tasks across 16 experienced open-source developers working in familiar repositories. With early-2025 AI tools allowed, task completion took 19% longer. Participants nevertheless believed AI had helped them work faster. That finding concerns those developers, tasks and tools; it cannot settle the value of every coding agent available today. [METR's original experiment](https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/). In February 2026, METR said its follow-up produced an unreliable estimate of current productivity effects. Developers increasingly declined work without AI or selected which tasks to submit; concurrent agents also complicated time measurement. The researchers considered greater acceleration plausible but said the data provided only weak evidence about its size. Repeating the old slowdown figure without this qualification would mislead readers. [METR's follow-up](https://metr.org/blog/2026-02-24-uplift-update/). For a manager, these studies justify a practical stance: keep the useful tools, investigate their effects and resist turning either enthusiasm or skepticism into a universal staffing formula. A team can benefit substantially on one class of work and lose time on another. Its delivery process needs to reveal that difference. ## Give each constraint its own remedy Before purchasing more CI capacity, map one change from its first proposed patch to production. Separate work that runs from work that waits. Identify the check that actually delays a decision, the reviewer whose attention is required and the release conditions that remain afterward. A simple timing record is often enough to expose a mistaken investment. If most delay occurs before review starts, shaving seconds off tests will not resolve it. If reviewers repeatedly wait for a long required check, improving that check has a direct purpose. If failures arise mainly after deployment, the team needs to examine what validation misses. Dependency caching illustrates why a familiar optimization needs measurement. GitHub documents caching as a way to reuse files that are costly to recreate on clean runners. It also says jobs must be able to recreate or download those files when a cache is unavailable. Cache behavior should therefore be assessed on both hits and misses. [GitHub's dependency-caching documentation](https://docs.github.com/en/actions/concepts/workflows-and-actions/dependency-caching). For the manager, the useful question is how predictable the feedback becomes. An excellent average can coexist with occasional waits long enough to disrupt a release. Track the slow end of the distribution, and distinguish a legitimate test failure from a failed download or exhausted runner. Each calls for a different response. Reliable, prompt feedback gives both people and agents an earlier opportunity to correct mistakes. Improving that shared infrastructure can benefit many kinds of development work, including changes written without AI. That gives managers a reason to fund it even while their choice of coding tools changes. ![One large stopwatch beside a task card contrasts with three separate worker trays, each with its own small clock.](/api/media/file/elapsed-aggregate-work-inline-v1.png) ## Faster tests still need to catch mistakes Some performance changes alter the conditions under which a test runs. That deserves closer scrutiny than a faster processor. Vitest, a JavaScript and TypeScript testing tool, isolates test files by default to keep one file's state from affecting another. Its documentation describes selective removal of isolation for suitable tests, while identifying mutable module state, modified globals and other dependencies as reasons to retain it. A setting that reduces preparation time can therefore change a correctness boundary. [Vitest's isolation guidance](https://vitest.dev/guide/recipes/disable-isolation). Linear used an opt-in approach for shared state, added cleanup and retained isolation for files it could not safely move. Its performance gain depended on deciding which tests could safely share that environment. [Linear's implementation](https://linear.app/now/ci-bottleneck-reworked). There is a separate challenge when an agent writes both a change and its tests. Consider a hypothetical discount rule that applies only to annual subscriptions. An agent could mistakenly implement it for monthly plans too, then write tests that confirm its own interpretation. Passing those tests would establish consistency with the mistake. A reviewer needs evidence anchored outside that implementation: the agreed requirement, examples of intended behavior and cases that should fail. Asking an agent for more tests may help, but a larger test count alone does not show that a misunderstood requirement has been challenged. The human responsibility here is to decide what the software should do and whether the evidence addresses that question. Teams that reward test volume without examining what the tests establish risk giving reviewers more material and less confidence. A manager should make time for that examination when assigning the work, rather than treating it as an unexplained delay after the agent has finished. ## Measure the change that reaches customers DORA's delivery metrics offer a useful downstream check. They include the time from committed code to production, deployment frequency, recovery after a failed deployment, the proportion of deployments requiring immediate intervention and the proportion representing unplanned incident-related rework. These describe delivery outcomes beyond a passing test run. [DORA's metric definitions](https://dora.dev/guides/dora-metrics/). For a team investigating its AI coding workflow, we would pair those outcomes with a small diagnostic record: - Feedback: elapsed time to the required checks, including slow runs and time waiting for capacity. - Consumption: aggregate runner work and repeated attempts associated with an accepted change. - Review: time awaiting attention, substantial revisions requested and the reason for each revision. - Customer impact: whether the released change delivered its intended behavior and needed corrective work. Use the record to investigate a particular workflow. A small documentation edit and a database migration should not be treated as interchangeable units of productivity. Where feasible, compare similar work over time and record other changes, such as a new runner provider or revised release process. An improvement occurring alongside AI adoption does not establish its cause. Managers can then make a concrete choice: fund the constraint that prevents useful changes from reaching customers with adequate confidence. Sometimes that will mean more computing capacity. Sometimes it will mean clearer requirements, better tests or enough experienced review time to understand the proposals already arriving. ## Sources - [Linear: AI coding and the CI bottleneck](https://linear.app/now/ci-bottleneck-reworked) — September 21, 2026. Mufeez Amjad's implementation account supplies the opening figures and selective-isolation example. First-party measurements describe Linear's system, not a controlled estimate of AI productivity or total spending. - [DORA: Balancing AI tensions](https://dora.dev/insights/balancing-ai-tensions/) — March 10, 2026. Separates the 2025 DORA association from a qualitative analysis of Google engineers' responses. That internal study reports experiences rather than measured causal effects; earlier questions may have primed responses. - [METR: Early-2025 AI and experienced developer productivity](https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/) — July 10, 2025 randomized experiment involving familiar open-source repositories and early-2025 tools. Supports the scoped slowdown result and perception discrepancy. It is not an estimate for all developers or September 2026 models. - [METR: Changing the developer productivity experiment](https://metr.org/blog/2026-02-24-uplift-update/) — February 24, 2026 update. Explains participant and task selection, changed compensation and problems measuring concurrent work. The researchers judge the new estimate unreliable; we do not present its raw figures as a current speedup. - [GitHub: Dependency caching](https://docs.github.com/en/actions/concepts/workflows-and-actions/dependency-caching) — Current documentation accessed September 22, 2026. Describes reuse on clean runners and the need for jobs to work without a cache. It does not establish the performance or bill for any particular repository. - [Vitest: Per-file isolation settings](https://vitest.dev/guide/recipes/disable-isolation) — Current documentation accessed September 22, 2026. Explains default isolation, selective configuration and state dependencies that require protection. Technical guidance supports the tradeoff; BIG CHANGE did not run a benchmark or change a test suite. - [DORA: Software delivery performance metrics](https://dora.dev/guides/dora-metrics/) — Updated January 5, 2026. Defines five delivery metrics and their application at a service level. These provide outcome measures beyond CI duration; the article's diagnostic examples are analysis, not results from a new experiment.