AWS Machine Learning Blog

OCTOBER 5, 2023

In this post, we walk through how to fine-tune Llama 2 on AWS Trainium , a purpose-built accelerator for LLM training, to reduce training times and costs. We review the fine-tuning scripts provided by the AWS Neuron SDK (using NeMo Megatron-LM), the various configurations we used, and the throughput results we saw.

AWS 125

AWS Machine Learning Machine Learning Deep Learning 125

AWS Machine Learning Blog

APRIL 25, 2024

Calculate a ROUGE-N score You can use the following steps to calculate a ROUGE-N score: Tokenize the generated summary and the reference summary into individual words or tokens using basic tokenization methods like splitting by whitespace or natural language processing (NLP) libraries.

AWS 132

AWS Algorithm Artificial Intelligence Artificial Intelligence 132

AWS Machine Learning Blog

NOVEMBER 9, 2023

In these two studies, commissioned by AWS, developers were asked to create a medical software application in Java that required use of their internal libraries. About the authors Qing Sun is a Senior Applied Scientist in AWS AI Labs and work on AWS CodeWhisperer, a generative AI-powered coding assistant.

AWS 108

AWS Natural Language Processing K-nearest Neighbors AI 108

AWS Machine Learning Blog

AUGUST 11, 2023

Natural language processing (NLP) is the field in machine learning (ML) concerned with giving computers the ability to understand text and spoken words in the same way as human beings can. Note that by following the steps in this section, you will deploy infrastructure to your AWS account that may incur costs.

Natural Language Processing 88

Natural Language Processing ML ML Machine Learning 88

DagsHub

JULY 25, 2024

TensorFlow The Google Brain team created the open-source deep learning framework TensorFlow, which was made available in 2015. Libraries and Extensions: Includes torchvision for image processing, touchaudio for audio processing, and torchtext for NLP. This environment supports collaborative development and experimentation.

Deep Learning 52

Deep Learning Deep Learning Machine Learning Machine Learning 52

AWS Machine Learning Blog

FEBRUARY 28, 2023

AWS provides the most complete set of services for the entire end-to-end data journey for all workloads, all types of data, and all desired business outcomes. The high-level steps involved in the solution are as follows: Use AWS Step Functions to orchestrate the health data anonymization pipeline.

ML 87

ML ML AWS Machine Learning 87

Pickl AI

AUGUST 22, 2024

In industry, it powers applications in computer vision, natural language processing, and reinforcement learning. This allows users to change the network architecture on-the-fly, which is particularly useful for tasks that require variable input sizes, such as natural language processing and reinforcement learning.

Deep Learning 52

Deep Learning Deep Learning Machine Learning Machine Learning 52