NVIDIA's October 23 post explains a tuner introduced in TensorRT 11.1: it compiles and measures alternative routes for an inference model. NVIDIA reports 1.14x–1.63x gains over its default route on ten workloads.
The big change
- What changed: Engineers can select a measured engine from routes examined for their model.
- Why it matters: The useful comparison is tied to the model, shapes, GPU and software stack. NVIDIA's gains are vendor results, not a transferable speed promise.
- What to watch: Accuracy filtering requires supplied inputs, reference outputs, a metric and a threshold. Without those checks, speed alone cannot establish acceptable output.
Before choosing an engine
The 11.4 manual, updated October 5, documents an ONNX input, a route expression, measured candidates and an optional saved engine. Search breadth changes build effort; no universal run cost is stated. The quickest eligible result is only the quickest route examined. The trtexec sample says tuning is unavailable on Windows.
The October post describes a Torch-TensorRT 2.15 Python implementation as planned. BIG CHANGE reviewed documentation and did not run a build. Our earlier GPU benchmark report concerns a separate workload comparison.
