Fine-tune GPT-OSS-20B for multilingual reasoning on HyperPod

AWS published a HyperPod walkthrough on 20 October 2026 for tuning GPT-OSS-20B to reason and answer in a requested language. Its most useful result is a distinction an ML team can test for itself: supervised fine-tuning (SFT) changed the language of the model's reasoning in AWS's small evaluation, while a subsequent GRPO run improved the language of some final answers. This guide turns the published sample into a decision exercise for an engineer with an existing SageMaker HyperPod cluster.

The numbers below are AWS-reported, not BIG CHANGE test results. The evaluation contains ten math and logic prompts translated into English, French, German, Spanish and Italian, for 50 model responses. It measures the detected language of the reasoning and final answer; it does not grade whether the answer to the problem is factually or mathematically right. A ten-case language slice is too small to establish performance across a language, domain or deployment.

Decide whether this run fits your environment

You need an EKS-orchestrated SageMaker HyperPod cluster, permission to submit GPU workloads, an installed HyperPod training operator and a Kueue gpu-queue. The sample SFT manifest requests four ml.g6e.12xlarge workers, each with four L40S GPUs, 180 GiB memory and one EFA resource. It selects that exact instance type and runs 16 processes. The GRPO and GRPO evaluation pods each request four GPUs and 180 GiB memory; the base and SFT inference pods used below each request four GPUs and 120 GiB. Check instance availability, account quotas, Kueue admission and capacity in your chosen Region before building an image. AWS used 16 L40S GPUs for its SFT result; substituting a smaller cluster changes the experiment and its runtime characteristics.

The manifests expect an FSx for Lustre persistent-volume claim called fsx-claim, mounted at /fsx, plus a working EFA, NCCL and aws-ofi-nccl stack for the multi-node SFT job. You also need an ECR repository, local AWS CLI, kubectl, Docker with buildx, envsubst, and access to the openai/gpt-oss-20b model and HuggingFaceH4/Multilingual-Thinking dataset. The model and dataset pages currently show Apache 2.0 licenses; review their current terms and your organization's data policy before use. The public dataset page shows a single train split of about 1,000 rows. AWS describes roughly 949 training and 50 validation examples in its own setup, which is distinct from the evaluation script's 50 fixed prompts.

Version used for these directions: AWS's 20 October 2026 article and the awslabs/awsome-distributed-ai repository at commit 240279b4, checked 21 October 2026. AWS's article still names an older 3.test_cases/... directory. The pinned repository puts the example in examples/training/trl/gpt-oss-lora-grpo/. Pin the revision you inspect; later changes may move files or alter defaults.

Run the sample and retain comparable outputs

1. Check the cluster and clone the pinned example. Confirm that kubectl addresses the intended EKS cluster and that its fsx-claim, queue and GPU nodes exist. The cluster, network, operator and storage are prerequisites, not resources this sample creates. From a Bash shell:

   git clone https://github.com/awslabs/awsome-distributed-ai.git
   cd awsome-distributed-ai
   git checkout 240279b4a8f9c12297c72ef1bee37ad3a2558884
   cd examples/training/trl/gpt-oss-lora-grpo
   aws eks update-kubeconfig --name <EKS_CLUSTER_NAME> --region <AWS_REGION>
   kubectl config current-context
   kubectl get pvc fsx-claim

2. Build the image, then inspect the rendered manifests. Copy env_vars.example to env_vars and set the account, ECR image, Region and Hugging Face token for your environment. The current README requires a shared trl-base:latest image before artifacts/build_push.sh, but its ../../../ build context is stale after the repository move. From this example directory, the shared TRL Dockerfile is one level up; the command below uses that path. The sample expands HF_TOKEN into pod environment values, so handle the rendered YAML and Kubernetes read access as credential-bearing. The evaluator uses deep-translator to request English translations of non-English reasoning from Google; a team using private inputs should review that data flow and adapt or disable it before running. Do not put private prompts into the sample unchanged.

   cp env_vars.example env_vars
   # Edit env_vars locally; do not commit credentials.
   source env_vars
   docker build -t trl-base:latest ..
   cd artifacts && ./build_push.sh && cd ..
   envsubst '$REGISTRY $IMAGE $TAG $HF_TOKEN' < kubernetes/fsx-storage-manager.yaml | kubectl apply -f -
   kubectl exec fsx-storage-manager -- mkdir -p /fsx/experiments

The image should appear in your ECR repository and fsx-storage-manager should start with /fsx mounted. If the image build or pull fails, check the base-image build, ECR registry/Region, image tag and node pull permissions before submitting training.

3. Run SFT and convert its checkpoint. The pinned manifest sets LoRA rank 8, 1,000 training steps, bf16, gradient checkpointing and FSDP full sharding. It writes checkpoints under /fsx/checkpoints. This is a costly multi-node job; confirm the queue admits all four workers and that the FSx volume is mounted on each one.

   envsubst '$REGISTRY $IMAGE $TAG $HF_TOKEN' < hyperpod-eks/train-lora-hyperpod-elastic-g6e.yaml | kubectl apply -f -
   kubectl get pods -l app=gpt-oss-lora-elastic
   kubectl logs -f -l app=gpt-oss-lora-elastic
   kubectl exec fsx-storage-manager -- ls /fsx/checkpoints

Once checkpoint-1000 actually exists, use the repository's converter:

   kubectl exec fsx-storage-manager -- python /app/convert_fsdp_checkpoint.py \
     --checkpoint /fsx/checkpoints/checkpoint-1000 \
     --output /fsx/checkpoints/converted-peft/lora-checkpoint-1000-peft

Keep the training logs and converted adapter. If a job stalls, inspect its pod events, queue admission, FSx mount and EFA/NCCL logs. Do not assume an elastic restart completed 1,000 steps merely because a pod is running; verify the checkpoint and final step.

4. Evaluate SFT before spending on GRPO. The pinned evaluate_grpo.py accepts --use_trained and --output, and its fixed prompt table covers ten questions in each of five languages. The sample's base and trained inference pods each provide four GPUs and the /fsx mount. Apply them one at a time if GPU capacity is tight; the commands below run the evaluator inside each pod and write separate reports. Wait for each pod to become ready before the exec command, and verify that your SFT adapter path exists.

   envsubst '$REGISTRY $IMAGE $TAG $HF_TOKEN' < kubernetes/inference-g6e-base.yaml | kubectl apply -f -
   kubectl exec inference-base -- python /app/evaluate_grpo.py \
     --output /fsx/experiments/base_eval.txt
   kubectl delete pod inference-base --ignore-not-found
   envsubst '$REGISTRY $IMAGE $TAG $HF_TOKEN' < kubernetes/inference-g6e-trained.yaml | kubectl apply -f -
   kubectl exec inference-trained -- python /app/evaluate_grpo.py \
     --use_trained \
     --sft_checkpoint /fsx/checkpoints/converted-peft/lora-checkpoint-1000-peft \
     --output /fsx/experiments/sft_eval.txt

Keep the base and SFT reports with the same evaluator and prompt table. The repository's supplied eval-grpo.yaml is configured for a later GRPO checkpoint; applying it unchanged now will point to a nonexistent adapter. Record the per-language reasoning and answer counts, and have speakers inspect ambiguous short answers and language-detector mistakes. Do not treat the script's answer-language field as answer correctness.

5. Run GRPO only if SFT leaves a language gap worth testing. The current GRPO manifest starts from /fsx/checkpoints/converted-peft/lora-checkpoint-1000-peft, samples eight generations per prompt and requests four GPUs. Confirm that path exists and inspect the manifest before applying it:

   envsubst '$REGISTRY $IMAGE $TAG $HF_TOKEN' < kubernetes/train-grpo-singlenode.yaml | kubectl apply -f -
   kubectl logs -f grpo-single
   kubectl exec fsx-storage-manager -- ls /fsx/checkpoints/grpo-singlenode

Select a checkpoint using held-out data, then convert it with /app/convert_grpo_checkpoint.py into a PEFT adapter. The sample's eval-grpo.yaml hardcodes /fsx/checkpoints/converted-peft/checkpoint-1470-peft; edit that evaluation path to the adapter your run actually produced. The inference-g6e-grpo.yaml pod starts sleep infinity and contains no checkpoint path. If you also use the README's example GRPO inference command, replace its hardcoded checkpoint path there. A step number from AWS's run is not a promised output of yours. Save the edited evaluation manifest and checkpoint identity alongside the result.

6. Run the same 50-case evaluation for the converted GRPO adapter. After replacing the hardcoded checkpoint path, apply the evaluation manifest and inspect its text report under /fsx/experiments/:

   envsubst '$REGISTRY $IMAGE $TAG $HF_TOKEN' < kubernetes/eval-grpo.yaml | kubectl apply -f -
   kubectl logs -f eval-grpo
   kubectl exec fsx-storage-manager -- ls /fsx/experiments

The report includes each prompt, extracted reasoning and final answer, detected languages, plus aggregate and per-language counts. Check all 50 rows, especially short numerical answers that a language detector may mark too_short or misclassify. For a deployment decision, add your own held-out tasks, human review and an answer-quality measure; the sample's 50 language checks cannot establish those outcomes.

Read the result before choosing SFT or GRPO

AWS reports the following on its own 50 responses. These are language-detection outcomes from one small evaluation, not independent measurements or broad model benchmarks.

The extra four compliant answers after GRPO are a useful lead for an experiment, with considerable sampling uncertainty. AWS's per-language final-answer count also moved unevenly: Spanish rose from 3/10 to 6/10, while Italian fell from 6/10 to 4/10. Inspect the rows and decide against the language and task mix you serve. If SFT already meets your measured requirement, stop there. If final-answer language still fails on a meaningful held-out set, test GRPO against the SFT adapter and watch for regressions. Keep answer correctness as a separate acceptance criterion.

Stop charges and protect the artifacts

Export only the checkpoints, logs and result files you are authorized to retain. Then remove the sample workloads:

kubectl delete hyperpodpytorchjob lora-hyperpod-elastic --ignore-not-found
kubectl delete pod grpo-single inference-base inference-trained inference-grpo eval-grpo fsx-storage-manager --ignore-not-found

Delete or scale down the HyperPod capacity according to your account's operating procedure, and remove FSx storage, ECR images and any S3 copies you no longer need. A pod deletion alone does not end cluster or storage charges. AWS's HyperPod prerequisites and pricing guidance directs users to check regional quotas and prices. Budget for GPU nodes during training, GRPO and evaluation, plus EKS, FSx, ECR and any retained data. Rates and availability vary by Region and account, so get a current estimate before provisioning. Keep access to model weights, dataset, checkpoint contents and any translated reasoning traces within the permissions approved for your project.

The big change

• For this five-language test, AWS's SFT run changed GPT-OSS-20B's measured reasoning-language compliance from 11/50 to 50/50. That supports trying SFT first when an observed language behavior is largely absent.
• AWS's GRPO run produced four more final answers in the requested language on the same 50 prompts, while one language slice declined. The next decision belongs to a held-out evaluation that reflects the actual users and tasks.
• The released sample makes the experiment inspectable, but its moved directory, stale base-image build path and hardcoded checkpoint require attention. Teams also need a 16-GPU SFT environment, explicit artifact handling and cleanup.

Sources & further reading

• AWS's 20 October 2026 HyperPod tutorial reports the experiment, infrastructure and results. AWS conducted it; BIG CHANGE did not reproduce it.
• Pinned AWS sample repository contains the current README, manifests, evaluator and checkpoint converters used to check the procedure.
• HyperPod EKS prerequisites and general prerequisites describe cluster setup, quotas, network and storage considerations.
• Model card and dataset card identify the public inputs and current access terms.