TorchNeuron public beta launches Oct 20, bringing native PyTorch support to AWS Trainium

AWS has announced that TorchNeuron, a PyTorch plugin, will be available as a public beta starting October 20, 2026. The plugin introduces a new native backend for AWS Trainium chips, supporting eager mode, torch.compile, and native distributed APIs such as FSDP and TP. Researchers and developers will be able to install it via pip install torch-neuronx.

Previously, running Hugging Face models on Trainium and Inferentia relied on the optimum-neuron library, which was built on PyTorch XLA and required compiler-style thinking with lazy tensors and static shapes. TorchNeuron changes this by integrating directly into PyTorch and exposing Neuron as a native device, allowing Transformers, Diffusers, Accelerate, PEFT, and TRL to run directly on Trainium without going through Optimum Neuron.

For inference, Transformers now natively supports text generation. TorchNeuron supports 89% of the top 100 most-used text generation model architectures out of the box, compared to 46% with Optimum Neuron v0.4.5. Users only need to pass device_map=neuron to move models to Neuron devices. Diffusers pipelines such as FLUX.1-dev, FLUX.2, and Qwen-Image are also compatible with tensor parallelism to maximize the use of Neuron instances.

For training, standard TRL fine-tuning scripts can run on Trainium instances without dedicated trainers or Neuron-specific model code. Parallelism is handled natively by Transformers via DistributedConfig, with tensor parallel and FSDP sizes set during model loading. The resulting checkpoints are in standard Transformers format and can be pushed to the Hugging Face Hub and loaded on other hardware.