AWS Machine Learning Blog

OCTOBER 5, 2023

In this post, we walk through how to fine-tune Llama 2 on AWS Trainium , a purpose-built accelerator for LLM training, to reduce training times and costs. We review the fine-tuning scripts provided by the AWS Neuron SDK (using NeMo Megatron-LM), the various configurations we used, and the throughput results we saw.

AWS 127

AWS Machine Learning Machine Learning Deep Learning 127

AWS Machine Learning Blog

APRIL 25, 2024

Calculate a ROUGE-N score You can use the following steps to calculate a ROUGE-N score: Tokenize the generated summary and the reference summary into individual words or tokens using basic tokenization methods like splitting by whitespace or natural language processing (NLP) libraries.

AWS 135

AWS Algorithm Artificial Intelligence Artificial Intelligence 135

AWS Machine Learning Blog

NOVEMBER 9, 2023

In these two studies, commissioned by AWS, developers were asked to create a medical software application in Java that required use of their internal libraries. About the authors Qing Sun is a Senior Applied Scientist in AWS AI Labs and work on AWS CodeWhisperer, a generative AI-powered coding assistant.

AWS 121

AWS Natural Language Processing K-nearest Neighbors Computer Science 121

AWS Machine Learning Blog

AUGUST 11, 2023

Natural language processing (NLP) is the field in machine learning (ML) concerned with giving computers the ability to understand text and spoken words in the same way as human beings can. Note that by following the steps in this section, you will deploy infrastructure to your AWS account that may incur costs.

Natural Language Processing 98

Natural Language Processing ML ML Machine Learning 98

AWS Machine Learning Blog

FEBRUARY 28, 2023

AWS provides the most complete set of services for the entire end-to-end data journey for all workloads, all types of data, and all desired business outcomes. The high-level steps involved in the solution are as follows: Use AWS Step Functions to orchestrate the health data anonymization pipeline.

ML 100

ML ML AWS Machine Learning 100

IBM Journey to AI blog

NOVEMBER 13, 2023

Kubernetes’s declarative, API -driven infrastructure has helped free up DevOps and other teams from manually driven processes so they can work more independently and efficiently to achieve their goals. And Kubernetes can scale ML workloads up or down to meet user demands, adjust resource usage and control costs.

Machine Learning 129

Machine Learning Machine Learning ML ML 129

DagsHub

JULY 25, 2024

TensorFlow The Google Brain team created the open-source deep learning framework TensorFlow, which was made available in 2015. Libraries and Extensions: Includes torchvision for image processing, touchaudio for audio processing, and torchtext for NLP. This environment supports collaborative development and experimentation.

Deep Learning 52

Deep Learning Deep Learning Machine Learning Machine Learning 52