AWS Machine Learning Blog

MAY 1, 2024

Llama2 by Meta is an example of an LLM offered by AWS. Llama 2 is an auto-regressive language model that uses an optimized transformer architecture and is intended for commercial and research use in English. Virginia) and US West (Oregon) AWS Regions, and most recently announced general availability in the US East (Ohio) Region.

AWS 105

AWS ML ML Clustering 105

AWS Machine Learning Blog

NOVEMBER 29, 2023

In today’s rapidly evolving landscape of artificial intelligence, deep learning models have found themselves at the forefront of innovation, with applications spanning computer vision (CV), natural language processing (NLP), and recommendation systems. use train_dataloader in the rest of the training logic.

Deep Learning 98

Deep Learning Deep Learning Natural Language Processing Artificial Intelligence 98

AWS Machine Learning Blog

JUNE 13, 2023

The size of the machine learning (ML) models––large language models ( LLMs ) and foundation models ( FMs )–– is growing fast year-over-year , and these models need faster and more powerful accelerators, especially for generative AI. With AWS Inferentia1, customers saw up to 2.3x With AWS Inferentia1, customers saw up to 2.3x

AWS 76

AWS ML ML Deep Learning 76

AWS Machine Learning Blog

MAY 10, 2023

In this post, we focus on how you can take advantage of the AWS Graviton3 -based Amazon Elastic Compute Cloud (EC2) C7g instances to help reduce inference costs by up to 50% relative to comparable EC2 instances for real-time inference on Amazon SageMaker. 4xlarge (AWS Graviton3) is about 50% of the c5.4xlarge and 40% of c6i.4xlarge;

AWS 84

AWS ML ML Machine Learning 84

AWS Machine Learning Blog

MAY 4, 2023

AWS has been innovating with purpose-built chips to address the growing need for powerful, efficient, and cost-effective compute hardware. You can use ml.trn1 and ml.inf2 compatible AWS Deep Learning Containers (DLCs) for PyTorch, TensorFlow, Hugging Face, and large model inference (LMI) to easily get started.

AWS 83

AWS Deep Learning Deep Learning ML 83

AWS Machine Learning Blog

JUNE 6, 2023

PyTorch is a machine learning (ML) framework that is widely used by AWS customers for a variety of applications, such as computer vision, natural language processing, content creation, and more. release, AWS customers can now do same things as they could with PyTorch 1.x 24xlarge with AWS PyTorch 2.0

AWS 73

AWS ML ML Deep Learning 73

AWS Machine Learning Blog

JUNE 26, 2024

In this post, we show how you can automate and intelligently process derivative confirms at scale using AWS AI services. An event notification on S3 object upload completion places a message in an Amazon Simple Queue Service (Amazon SQS) queue to invoke an AWS Lambda function.

AWS 102

AWS ML ML AI 102

AWS Machine Learning Blog

NOVEMBER 22, 2023

Yes, the AWS re:Invent season is upon us and as always, the place to be is Las Vegas! Now all you need is some guidance on generative AI and machine learning (ML) sessions to attend at this twelfth edition of re:Invent. are the sessions dedicated to AWS DeepRacer ! Generative AI is at the heart of the AWS Village this year.

AWS 124

AWS ML ML AI 124

AWS Machine Learning Blog

MAY 31, 2024

Genomic language models are a new and exciting field in the application of large language models to challenges in genomics. In this blog post and open source project , we show you how you can pre-train a genomics language model, HyenaDNA , using your genomic data in the AWS Cloud.

AWS 97

AWS ML ML Machine Learning 97

AWS Machine Learning Blog

JULY 26, 2023

In this post, we show how you can run Stable Diffusion models and achieve high performance at the lowest cost in Amazon Elastic Compute Cloud (Amazon EC2) using Amazon EC2 Inf2 instances powered by AWS Inferentia2. versions on AWS Inferentia2 cost-effectively. You can run both Stable Diffusion 2.1 The Stable Diffusion 2.1

AWS 85

AWS Deep Learning Deep Learning ML 85

Julien Simon

FEBRUARY 24, 2023

Video — Transformer training shootout: AWS Trainium vs. NVIDIA A10G In this video, I compare the cost-performance of AWS Trainium, a new custom chip designed by AWS, with NVIDIA A10G GPUs. Then, I run a natural language processing job, fine-tuning the BERT Large model on the full Yelp review datatset.

AWS 40

AWS Natural Language Processing Deep Learning Deep Learning 40

Data Science 101

DECEMBER 27, 2019

AWS Deep Learning Containers now support Tensorflow 2.0 AWS Deep Learning Containers are docker images which are preconfigured for deep learning tasks. Build a custom classifier using AWS Comprehend AWS Comprehend is a Natural Language Processing (NLP) service.

Cloud Data 86

Cloud Data Data Science Deep Learning Deep Learning 86

AWS Machine Learning Blog

SEPTEMBER 9, 2024

As LLMs have grown larger, their performance on a wide range of natural language processing tasks has also improved significantly, but the increased size of LLMs has led to significant computational and resource challenges. AWS is the first leading cloud provider to offer the H200 GPU in production.

AWS 101

AWS Deep Learning Deep Learning ML 101

Julien Simon

FEBRUARY 23, 2023

In this video, I show you how to accelerate Transformer training with AWS Trainium, a new custom chip designed by AWS, and the brand new Hugging Face Neuron Amazon Machine Image (AMI). Then, I run a natural language processing job, accelerating a BERT model to classify the Yelp review datatset on 32 Neuron cores.

AWS 40

AWS Natural Language Processing Deep Learning Deep Learning 40

AWS Machine Learning Blog

SEPTEMBER 16, 2024

DNABERT 6 Dataset For this post, we use the gRNA data released by researchers in a paper about gRNA prediction using deep learning. In this section, we describe the process we followed to create the training and evaluation datasets for this task. CRISPRon is a CNN based deep learning model.

Natural Language Processing 111

Natural Language Processing AWS Deep Learning Deep Learning 111

AWS Machine Learning Blog

JUNE 18, 2024

Some examples include extracting players and positions in an NFL game summary, products mentioned in an AWS keynote transcript, or key names from an article on a favorite tech company. This process must be repeated for every new document and entity type, making it impractical for processing large volumes of documents at scale.

AWS 116

AWS Natural Language Processing Machine Learning Machine Learning 116

AWS Machine Learning Blog

JANUARY 13, 2023

FL doesn’t require moving or sharing data across sites or with a centralized server during the model training process. In this two-part series, we demonstrate how you can deploy a cloud-based FL framework on AWS. Participants can either choose to maintain their data in their on-premises systems or in an AWS account that they control.

AWS 81

AWS Analytics Analytics Machine Learning 81

AWS Machine Learning Blog

DECEMBER 14, 2023

In line with this mission, Talent.com collaborated with AWS to develop a cutting-edge job recommendation engine driven by deep learning, aimed at assisting users in advancing their careers. The solution does not require porting the feature extraction code to use PySpark, as required when using AWS Glue as the ETL solution.

ETL 96

ETL AWS ML ML 96

ODSC - Open Data Science

FEBRUARY 17, 2023

Natural language processing (NLP) has been growing in awareness over the last few years, and with the popularity of ChatGPT and GPT-3 in 2022, NLP is now on the top of peoples’ minds when it comes to AI. Java has numerous libraries designed for the language, including CoreNLP, OpenNLP, and others.

Deep Learning 111

Deep Learning Deep Learning Data Science Natural Language Processing 111

Data Science 101

NOVEMBER 29, 2019

Use Amazon Sagemaker to add ML predictions in Amazon QuickSight Amazon QuickSight, the AWS BI tool, now has the capability to call Machine Learning models. Amazon Comprehend launches real-time classification Amazon Comprehend is a service which uses Natural Language Processing (NLP) to examine documents.

Cloud Data 115

Cloud Data Data Science Machine Learning Machine Learning 115

AWS Machine Learning Blog

OCTOBER 23, 2023

Given this mission, Talent.com and AWS joined forces to create a job recommendation engine using state-of-the-art natural language processing (NLP) and deep learning model training techniques with Amazon SageMaker to provide an unrivaled experience for job seekers. The recommendation system has driven an 8.6%

AWS 104

AWS Deep Learning Deep Learning Machine Learning 104

AWS Machine Learning Blog

SEPTEMBER 16, 2024

RAG is an approach that combines information retrieval techniques with natural language processing (NLP) to enhance the performance of text generation or language modeling tasks. This method involves retrieving relevant information from a large corpus of text data and using it to augment the generation process.

AWS 105

AWS Deep Learning Deep Learning AI 105

Dataconomy

JUNE 16, 2023

The cloud-based DLP solution from Gamma AI uses cutting-edge deep learning for contextual perception to achieve a data classification accuracy of 99.5%. They do this by utilizing machine learning and natural language processing. Click “ See it in action ” and wait for the demo.

AI 193

AI AI Natural Language Processing Data Classification 193

AWS Machine Learning Blog

OCTOBER 30, 2024

Global Resiliency is a new Amazon Lex capability that enables near real-time replication of your Amazon Lex V2 bots in a second AWS Region. We showcase the replication process of bot versions and aliases across multiple Regions. Solution overview For this exercise, we create a BookHotel bot as our sample bot.

AWS 94

AWS AI AI Natural Language Processing 94

AWS Machine Learning Blog

APRIL 1, 2024

Machine learning (ML) research has proven that large language models (LLMs) trained with significantly large datasets result in better model quality. The following figure shows how FSDP works for two data parallel processes. In the following sections, we explain the end-to-end process in more detail.

Clustering 120

Clustering AWS ML ML 120

AWS Machine Learning Blog

APRIL 25, 2024

We provide a comprehensive guide on how to deploy speaker segmentation and clustering solutions using SageMaker on the AWS Cloud. Solution overview Amazon Transcribe is the go-to service for speaker diarization in AWS. Hugging Face is a popular open source hub for machine learning (ML) models.

AWS 111

AWS ML ML Python 111

AWS Machine Learning Blog

JUNE 15, 2023

Dive into Deep Learning ( D2L.ai ) is an open-source textbook that makes deep learning accessible to everyone. Then we build Jupyter notebooks in SageMaker to run the translation process using Amazon Translate public APIs. About the authors Yunfei Bai is a Senior Solutions Architect at AWS.

AWS 83

AWS Natural Language Processing Deep Learning Deep Learning 83

AWS Machine Learning Blog

FEBRUARY 12, 2024

One of the several challenges faced was adapting the existing on-premises pipeline solution for use on AWS. The solution involved two key components: Modifying and extending existing code – The first part of our solution involved the modification and extension of our existing code to make it compatible with AWS infrastructure.

ML 125

ML ML AWS Machine Learning 125

AWS Machine Learning Blog

NOVEMBER 27, 2023

of Large Model Inference (LMI) Deep Learning Containers (DLCs) and adds support for NVIDIA’s TensorRT-LLM Library. The task parameter is used to define the natural language processing (NLP) task. Qing Lan is a Software Development Engineer in AWS. Today, Amazon SageMaker launches a new version (0.25.0)

AWS 106

AWS Deep Learning Deep Learning Machine Learning 106

AWS Machine Learning Blog

OCTOBER 7, 2024

By using the natural language processing and generation capabilities of generative AI, the chat assistant can understand user queries, retrieve relevant information from various data sources, and provide tailored, contextual responses. When selecting websites to index, adhere to the AWS Acceptable Use Policy and other AWS terms.

AWS 94

AWS AI AI Natural Language Processing 94

Pickl AI

AUGUST 22, 2024

Introduction Deep Learning frameworks are crucial in developing sophisticated AI models, and driving industry innovations. By understanding their unique features and capabilities, you’ll make informed decisions for your Deep Learning applications.

Deep Learning 52

Deep Learning Deep Learning Machine Learning Machine Learning 52

AWS Machine Learning Blog

NOVEMBER 22, 2023

Amazon Elastic Compute Cloud (Amazon EC2) DL2q instances, powered by Qualcomm AI 100 Standard accelerators, can be used to cost-efficiently deploy deep learning (DL) workloads in the cloud. To learn more about onboarding and compiling models on the DL2q instance, see the Cloud AI100 Tutorial Documentation. Roy from Qualcomm AI.

AI 118

AI AI AWS Deep Learning 118

Pickl AI

JULY 12, 2024

Summary: This guide explores Artificial Intelligence Using Python, from essential libraries like NumPy and Pandas to advanced techniques in machine learning and deep learning. TensorFlow and Keras: TensorFlow is an open-source platform for machine learning.

Artificial Intelligence 52

Artificial Intelligence Artificial Intelligence Python Natural Language Processing 52

AWS Machine Learning Blog

MAY 31, 2024

Falcon 2 11B is supported by the SageMaker TGI Deep Learning Container (DLC) which is powered by Text Generation Inference (TGI) , an open source, purpose-built solution for deploying and serving LLMs that enables high-performance text generation using tensor parallelism and dynamic batching. Avan Bala is a Solutions Architect at AWS.

AWS 111

AWS Python ML ML 111

AWS Machine Learning Blog

JUNE 13, 2023

In this post, we demonstrate how to deploy Falcon for applications like language understanding and automated writing assistance using large model inference deep learning containers on SageMaker. SageMaker large model inference (LMI) deep learning containers (DLCs) can help.

Deep Learning 80

Deep Learning Deep Learning AWS Machine Learning 80

AWS Machine Learning Blog

APRIL 18, 2023

MLSL’s high caliber talent, culture, and focus on aiding our realization of measurable and compelling results from machine learning investments enabled us to reduce suicide risk, improve career transition, and speed up important connections for our service members, veterans, and their families.” Applied AI Specialist Architect at AWS.

AWS 76

AWS Machine Learning Machine Learning ML 76

Mlearning.ai

JUNE 14, 2023

Learning LLMs (Foundational Models) Base Knowledge / Concepts: What is AI, ML and NLP Introduction to ML and AI — MFML Part 1 — YouTube What is NLP (Natural Language Processing)? — YouTube YouTube Introduction to Natural Language Processing (NLP) NLP 2012 Dan Jurafsky and Chris Manning (1.1)

Natural Language Processing 52

Natural Language Processing ML ML Support Vector Machines 52