Frugality meets Accuracy: Cost-efficient training of GPT NeoX and Pythia models with AWS Trainium
AWS Machine Learning Blog
DECEMBER 12, 2023
In this post, we’ll summarize training procedure of GPT NeoX on AWS Trainium , a purpose-built machine learning (ML) accelerator optimized for deep learning training. Training steps To run the training, we use SLURM managed multi-node Amazon Elastic Compute Cloud ( Amazon EC2 ) Trn1 cluster, with each node containing a trn1.32xl instance.
Let's personalize your content