# AWS coding-agent routing sample

> Experimental routing needs whole-task evaluation.

By BIGCHANGE.AI Editorial

Published: 2026-11-14T19:51:38.346Z
Updated: 2026-11-14T19:51:38.346Z
Canonical: https://bigchange.ai/blog/bedrock-switchyard-agent-routing-sample

![Illustration of a developer reviewing a coding task.](https://bigchange.ai/api/media/file/switchyard-v2.png)
AI-generated illustration for BIG CHANGE.

AWS's [November 13 sample](https://aws.amazon.com/blogs/machine-learning/route-coding-agent-requests-to-the-right-model-on-amazon-bedrock-with-nvidia-nemo-switchyard/) uses a LiteLLM gateway and NVIDIA Switchyard to route coding-agent turns to Bedrock models. It is experimental, non-production code.

## The big change

- AWS supplies a comparison method, not demonstrated savings. Failed attempts count toward cost even when they do not produce a solved task.
- NVIDIA separates routing decisions from the software that calls models. Choosing a destination does not itself manage credentials, retries or the surrounding service.
- LiteLLM's key-management documentation separates model access from administrative permissions. A gateway credential needs an appropriate scope, not simply a valid format.

## What the sample measures

Stage examines recent tool results to choose between Nemotron 3 Super and Claude Opus 5.5; Random uses configured weights.

The runner's slug task repairs a URL-building function. Tests passing within the turn limit count as success. It compares routed runs with fixed-model baselines, dividing all estimated attempt costs by successful runs. Unpriced runs remain unknown; estimates are not bills.

Stage can switch after failures, replaying accumulated history and potentially costing more than starting with the stronger candidate. The small tasks establish no general cost or quality advantage.

The runner executes generated Python without an operating-system sandbox, with the user's file and network access. AWS recommends disposable development environments. Even local gateway tests make billable Bedrock calls.

## Routing and running are different jobs

[NVIDIA's project documentation](https://github.com/NVIDIA-NeMo/Switchyard) describes a library embedded in a gateway or agent harness: the software surrounding the model. The application supplies a model pool, request and state. Switchyard chooses a route; the application's clients perform the model calls and retain responsibility for credentials and retries.

Its Execution, also called Stage, approach reads recent tool activity and outcomes, so selection can change as an agent works. Task routing instead asks a model to judge whether an efficient candidate can handle the request. These names describe different decision methods, not a guarantee of success. Integration support varies, so the full catalog does not imply every gateway can run every method.

NVIDIA labels its embedded core beta, its model client and runner alpha, and its standalone server a demo for local evaluation rather than a supported production deployment. APIs and configuration can change. A team therefore needs to evaluate the whole agent, model pool and configuration against a fixed-model baseline, rather than transferring a result between different setups.

## A gateway also has an access boundary

[LiteLLM's virtual-key documentation](https://docs.litellm.ai/docs/proxy/virtual_keys) describes separate controls for model access and management routes. A key's owner is not necessarily its creator. Model permissions are evaluated against the key and its applicable constraints; administrative-route permissions depend on the owner's role, with explicit route restrictions available.

That distinction matters because permission to request an answer and permission to manage a gateway are different powers. Its documentation also describes recording estimated spending by key, user and team using model-cost data. Such a record helps attribute use; it does not establish that an answer passed a task's acceptance test. These general controls do not establish which features a particular deployment exposes.

BIG CHANGE reviewed documentation and did not deploy the sample, execute generated code or make model calls.

## Sources

- [AWS routing sample](https://aws.amazon.com/blogs/machine-learning/route-coding-agent-requests-to-the-right-model-on-amazon-bedrock-with-nvidia-nemo-switchyard/) — November 13 sample.
- [NVIDIA Switchyard](https://github.com/NVIDIA-NeMo/Switchyard) — Routing library and components.
- [LiteLLM virtual keys](https://docs.litellm.ai/docs/proxy/virtual_keys) — Key permissions and tracking.
