AWS Machine Learning Blog

OCTOBER 10, 2023

Customers often need to train a model with data from different regions, organizations, or AWS accounts. Existing partner open-source FL solutions on AWS include FedML and NVIDIA FLARE. These open-source packages are deployed in the cloud by running in virtual machines, without using the cloud-native services available on AWS.

AWS 111

AWS ML ML Algorithm 111

AWS Machine Learning Blog

OCTOBER 5, 2023

In this post, we walk through how to fine-tune Llama 2 on AWS Trainium , a purpose-built accelerator for LLM training, to reduce training times and costs. We review the fine-tuning scripts provided by the AWS Neuron SDK (using NeMo Megatron-LM), the various configurations we used, and the throughput results we saw.

AWS 116

AWS Machine Learning Machine Learning Deep Learning 116

AWS Machine Learning Blog

OCTOBER 31, 2024

The web application that the user uses to retrieve answers is connected to an identity provider (IdP) or AWS IAM Identity Center. If you haven’t created one yet, refer to Build private and secure enterprise generative AI apps with Amazon Q Business and AWS IAM Identity Center for instructions. Access to AWS Secrets Manager.

AWS 110

AWS AI AI ML 110

AWS Machine Learning Blog

MAY 1, 2024

Llama2 by Meta is an example of an LLM offered by AWS. To learn more about Llama 2 on AWS, refer to Llama 2 foundation models from Meta are now available in Amazon SageMaker JumpStart. Virginia) and US West (Oregon) AWS Regions, and most recently announced general availability in the US East (Ohio) Region.

AWS 107

AWS ML ML Clustering 107

AWS Machine Learning Blog

APRIL 30, 2024

To address these issues, we launched a generative artificial intelligence (AI) call summarization feature in Amazon Transcribe Call Analytics. You can also use generative call summarization through Amazon Transcribe Post Call Analytics Solution for post-call summaries. This reduces customer wait times and improves agent productivity.

Analytics 113

Analytics Analytics AI AI 113

Flipboard

DECEMBER 11, 2024

Data engineers use data warehouses, data lakes, and analytics tools to load, transform, clean, and aggregate data. SageMaker Unied Studio is an integrated development environment (IDE) for data, analytics, and AI. As AI and analytics use cases converge, transform how data teams work together with SageMaker Unified Studio.

SQL 160

SQL AWS Data Lakes AI 160

AWS Machine Learning Blog

NOVEMBER 30, 2023

The number of companies launching generative AI applications on AWS is substantial and building quickly, including adidas, Booking.com, Bridgewater Associates, Clariant, Cox Automotive, GoDaddy, and LexisNexis Legal & Professional, to name just a few. Innovative startups like Perplexity AI are going all in on AWS for generative AI.

AWS 137

AWS AI AI ML 137

AWS Machine Learning Blog

MARCH 15, 2024

Therefore, ML creates challenges for AWS customers who need to ensure privacy and security across distributed entities without compromising patient outcomes. After a blueprint is configured, it can be used to create consistent environments across multiple AWS accounts and Regions using continuous deployment automation.

AWS 118

AWS ML ML Machine Learning 118

AWS Machine Learning Blog

MAY 31, 2024

In this blog post and open source project , we show you how you can pre-train a genomics language model, HyenaDNA , using your genomic data in the AWS Cloud. It supports large-scale analysis and collaborative research through HealthOmics storage, analytics, and workflow capabilities.

AWS 99

AWS ML ML Machine Learning 99

AWS Machine Learning Blog

OCTOBER 16, 2024

With these hyperlinks, we can bypass traditional memory and storage-intensive methods of first downloading and subsequently processing images locally—a task made even more daunting by the size and scale of our dataset, spanning over 4 TB. About the Author Xiong Zhou is a Senior Applied Scientist at AWS.

ML 101

ML ML Clustering Machine Learning 101

AWS Machine Learning Blog

MAY 4, 2023

AWS has been innovating with purpose-built chips to address the growing need for powerful, efficient, and cost-effective compute hardware. You can use ml.trn1 and ml.inf2 compatible AWS Deep Learning Containers (DLCs) for PyTorch, TensorFlow, Hugging Face, and large model inference (LMI) to easily get started. petaflops for BF16/FP16.

AWS 88

AWS Deep Learning Deep Learning ML 88

AWS Machine Learning Blog

NOVEMBER 29, 2024

You can use open-source libraries, or the AWS managed Large Model Inference (LMI) deep learning container (DLC) to dynamically load and unload adapter weights. Prerequisites To run the example notebooks, you need an AWS account with an AWS Identity and Access Management (IAM) role with permissions to manage resources created.

AWS 91

AWS ML ML Machine Learning 91

AWS Machine Learning Blog

JUNE 11, 2024

Sprinklr’s specialized AI models streamline data processing, gather valuable insights, and enable workflows and analytics at scale to drive better decision-making and productivity. During this journey, we collaborated with our AWS technical account manager and the Graviton software engineering teams.

Machine Learning 97

Machine Learning Machine Learning AWS Natural Language Processing 97

AWS Machine Learning Blog

MARCH 8, 2023

This post shows a way to do this using Snowflake as the data source and by downloading the data directly from Snowflake into a SageMaker Training job instance. We create a custom training container that downloads data directly from the Snowflake table into the training instance rather than first downloading the data into an S3 bucket.

ML 124

ML ML AWS Python 124

AWS Machine Learning Blog

MARCH 15, 2024

In addition to Amazon Bedrock, you can use other AWS services like Amazon SageMaker JumpStart and Amazon Lex to create fully automated and easily adaptable generative AI order processing agents. In this post, we show you how to build a speech-capable order processing agent using Amazon Lex, Amazon Bedrock, and AWS Lambda.

AWS 135

AWS AI AI Python 135

AWS Machine Learning Blog

APRIL 7, 2023

As an example, smart venue solutions can use near-real-time computer vision for crowd analytics over 5G networks, all while minimizing investment in on-premises hardware networking equipment. In this post, we demonstrate how to deploy a SageMaker model to AWS Wavelength to reduce model inference latency for 5G network-based applications.

AWS 92

AWS Clustering ML ML 92

AWS Machine Learning Blog

AUGUST 30, 2023

Powered by Amazon Lex , the QnABot on AWS solution is an open-source, multi-channel, multi-language conversational chatbot. This is why QnABot also integrates with any other LLM using an AWS Lambda function that you provide. QnABot can retrieve relevant passages from an Amazon Kendra index (containing AWS documentation).

AWS 122

AWS AI AI Analytics 122

AWS Machine Learning Blog

APRIL 18, 2024

It’s straightforward to deploy in your AWS account. Prerequisites You need to have an AWS account and an AWS Identity and Access Management (IAM) role and user with permissions to create and manage the necessary resources and components for this application. Everything you need is provided as open source in our GitHub repo.

AWS 126

AWS Analytics Analytics AI 126

AWS Machine Learning Blog

JANUARY 22, 2024

When the image gets saved in the S3 bucket, it invokes an AWS Step Functions workflow: The Queries-Decider AWS Lambda function examines the document passed in and adds information about the mime type, the number of pages, and the number of queries to the Step Functions workflow (for our example, we have four queries).

AWS 117

AWS ML ML Machine Learning 117

AWS Machine Learning Blog

FEBRUARY 29, 2024

At Deutsche Bahn, a dedicated AI platform team manages and operates the SageMaker Studio platform, and multiple data analytics teams within the organization use the platform to develop, train, and run various analytics and ML activities. The AD group contains scientists who needs access to their team’s SageMaker domain.

AWS 117

AWS AI AI Data Scientist 117

AWS Machine Learning Blog

MARCH 1, 2024

Machine learning (ML) technologies continually improve and power the contact center customer experience by providing solutions for capabilities like self-service bots, live call analytics, and post-call analytics. Set up an AWS account with the appropriate permissions. For instructions, refer to Installing the AWS SAM CLI.

AWS 113

AWS ML ML Analytics 113

Flipboard

AUGUST 17, 2023

Amazon Redshift uses SQL to analyze structured and semi-structured data across data warehouses, operational databases, and data lakes, using AWS-designed hardware and ML to deliver the best price-performance at any scale. Prerequisites To continue with the examples in this post, you need to create the required AWS resources.

ML 123

ML ML AWS Data Warehouse 123

AWS Machine Learning Blog

APRIL 17, 2024

With AWS, you can deploy this solution in a serverless, scalable, and fully event-driven architecture. This post demonstrates a proof of concept built on two key AWS services well suited for graph knowledge representation and natural language processing: Amazon Neptune and Amazon Bedrock.

AWS 127

AWS Database Natural Language Processing AI 127

AWS Machine Learning Blog

SEPTEMBER 20, 2023

For example, you might have acquired a company that was already running on a different cloud provider, or you may have a workload that generates value from unique capabilities provided by AWS. We show how you can build and train an ML model in AWS and deploy the model in another platform.

ML 118

ML ML Azure AWS 118

AWS Machine Learning Blog

APRIL 16, 2024

IAM role – SageMaker requires an AWS Identity and Access Management (IAM) role to be assigned to a SageMaker Studio domain or user profile to manage permissions effectively. Create database connections The built-in SQL browsing and execution capabilities of SageMaker Studio are enhanced by AWS Glue connections. or later image versions.

SQL 108

SQL AWS Database Data Scientist 108

AWS Machine Learning Blog

FEBRUARY 28, 2024

The solution in this post aims to bring enterprise analytics operations to the next level by shortening the path to your data using natural language. Our solution aims to address those challenges using Amazon Bedrock and AWS Analytics Services. Install the AWS Command Line Interface (AWS CLI).

SQL 126

SQL AWS Database ML 126

AWS Machine Learning Blog

APRIL 19, 2024

We used AWS services including Amazon Bedrock , Amazon SageMaker , and Amazon OpenSearch Serverless in this solution. In this series, we use the slide deck Train and deploy Stable Diffusion using AWS Trainium & AWS Inferentia from the AWS Summit in Toronto, June 2023 to demonstrate the solution.

AWS 120

AWS ML ML Database 120

AWS Machine Learning Blog

MARCH 25, 2024

Amazon Transcribe is an AWS service that allows customers to convert speech to text in either batch or streaming mode. AWS is responsible for protecting the global infrastructure that runs all of the AWS Cloud. This content includes the security configuration and management tasks for the AWS services that you use.

AWS 97

AWS Machine Learning Machine Learning Natural Language Processing 97

AWS Machine Learning Blog

APRIL 25, 2024

We provide a comprehensive guide on how to deploy speaker segmentation and clustering solutions using SageMaker on the AWS Cloud. Solution overview Amazon Transcribe is the go-to service for speaker diarization in AWS. Make sure the AWS account has a service quota for hosting a SageMaker endpoint for an ml.g5.2xlarge instance.

AWS 113

AWS ML ML Python 113

AWS Machine Learning Blog

DECEMBER 15, 2023

In this post, we discuss how to bring data stored in Amazon DocumentDB into SageMaker Canvas and use that data to build ML models for predictive analytics. You want to gather insights on this data and build an ML model to predict how new restaurants will be rated, but find it challenging to perform analytics on unstructured data.

Machine Learning 106

Machine Learning Machine Learning AWS ML 106

AWS Machine Learning Blog

JANUARY 30, 2024

and AWS services including Amazon Bedrock and Amazon SageMaker to perform similar generative tasks on multimodal data. In this post, we use the slide deck titled Train and deploy Stable Diffusion using AWS Trainium & AWS Inferentia from the AWS Summit in Toronto, June 2023, to demonstrate the solution. The model.tar.gz

AWS 117

AWS ML ML K-nearest Neighbors 117

AWS Machine Learning Blog

SEPTEMBER 6, 2023

The Amazon Kendra AEM connector can integrate with AWS IAM Identity Center (Successor to AWS Single Sign-On). Prerequisites To try out the Amazon Kendra connector for AEM using this post as a reference, you need the following: An AWS account with privileges to create AWS Identity and Access Management (IAM) roles and policies.

AWS 115

AWS Analytics Analytics Cloud Computing 115

AWS Machine Learning Blog

AUGUST 1, 2023

With the power of state-of-the-art techniques, the creative agency can support their customer by using generative AI models within their secure AWS environment. AWS has also developed hardware and chips using AWS Inferentia2 for high performance at the lowest cost for generative AI inference. to the local directory as tar.gz

AWS 69

AWS ML ML AI 69

AWS Machine Learning Blog

NOVEMBER 26, 2023

Amazon Personalize is excited to announce the new Next Best Action ( aws-next-best-action ) recipe to help you determine the best actions to suggest to your individual users that will enable you to increase brand loyalty and conversion.

AWS 95

AWS ML ML Machine Learning 95

AWS Machine Learning Blog

SEPTEMBER 12, 2024

Prerequisites To implement this solution, you need the following: An AWS account with privileges to create AWS Identity and Access Management (IAM) roles and policies. Basic familiarity with SageMaker and AWS services that support LLMs. For more information, see Overview of access management: Permissions and policies.

AWS 125

AWS ML ML AI 125

AWS Machine Learning Blog

OCTOBER 9, 2024

The service, which was launched in March 2021, predates several popular AWS offerings that have anomaly detection, such as Amazon OpenSearch , Amazon CloudWatch , AWS Glue Data Quality , Amazon Redshift ML , and Amazon QuickSight. To use this feature, you can write rules or analyzers and then turn on anomaly detection in AWS Glue ETL.

AWS 83

AWS ML ML Data Quality 83

AWS Machine Learning Blog

APRIL 3, 2024

Prerequisites Before you dive into the solution, make sure you have signed up for and created an AWS account. canvas.ipynb After the notebook is downloaded, you can open the notebook and run the notebook to evaluate further. Meenakshisundaram Thandavarayan works for AWS as an AI/ ML Specialist.

Machine Learning 117

Machine Learning Machine Learning ML ML 117