AWS Machine Learning Blog

SEPTEMBER 18, 2024

The compute clusters used in these scenarios are composed of more than thousands of AI accelerators such as GPUs or AWS Trainium and AWS Inferentia , custom machine learning (ML) chips designed by Amazon Web Services (AWS) to accelerate deep learning workloads in the cloud.

Clustering 116

Clustering AWS ML ML 116

AWS Machine Learning Blog

NOVEMBER 19, 2024

In 2018, I sat in the audience at AWS re:Invent as Andy Jassy announced AWS DeepRacer —a fully autonomous 1/18th scale race car driven by reinforcement learning. At the time, I knew little about AI or machine learning (ML). seconds, securing the 2018 AWS DeepRacer grand champion title!

AWS 105

AWS ML ML AI 105

AWS Machine Learning Blog

JULY 31, 2024

Amazon Web Services (AWS) is committed to supporting the development of cutting-edge generative artificial intelligence (AI) technologies by companies and organizations across the globe. Let’s dive in and explore how these organizations are transforming what’s possible with generative AI on AWS.

AWS 126

AWS AI AI Clustering 126

AWS Machine Learning Blog

JANUARY 24, 2024

Generative AI solutions have the potential to transform businesses by boosting productivity and improving customer experiences, and using large language models (LLMs) with these solutions has become increasingly popular. Despite their wealth of general knowledge, state-of-the-art LLMs only have access to the information they were trained on.

AWS 144

AWS Database AI AI 144

AWS Machine Learning Blog

MAY 1, 2024

Large language models (LLMs) are making a significant impact in the realm of artificial intelligence (AI). Llama2 by Meta is an example of an LLM offered by AWS. To learn more about Llama 2 on AWS, refer to Llama 2 foundation models from Meta are now available in Amazon SageMaker JumpStart.

AWS 122

AWS ML ML Clustering 122

AWS Machine Learning Blog

MAY 30, 2024

Generative artificial intelligence (AI) has unlocked fresh opportunities for these use cases. In this post, we introduce the Media Analysis and Policy Evaluation solution, which uses AWS AI and generative AI services to provide a framework to streamline video extraction and evaluation processes.

AWS 127

AWS AI AI ML 127

AWS Machine Learning Blog

SEPTEMBER 4, 2024

A challenge for DevOps engineers is the additional complexity that comes from using Kubernetes to manage the deployment stage while resorting to other tools (such as the AWS SDK or AWS CloudFormation ) to manage the model building pipeline. kubectl for working with Kubernetes clusters. eksctl for working with EKS clusters.

AWS 107

AWS Clustering ML ML 107

AWS Machine Learning Blog

JUNE 25, 2024

Amazon Web Services is excited to announce the launch of the AWS Neuron Monitor container , an innovative tool designed to enhance the monitoring capabilities of AWS Inferentia and AWS Trainium chips on Amazon Elastic Kubernetes Service (Amazon EKS). The Container Insights dashboard also shows cluster status and alarms.

AWS 94

AWS ML ML Clustering 94

Flipboard

JUNE 20, 2023

For reference, GPT-3, an earlier generation LLM has 175 billion parameters and requires months of non-stop training on a cluster of thousands of accelerated processors. The Carbontracker study estimates that training GPT-3 from scratch may emit up to 85 metric tons of CO2 equivalent, using clusters of specialized hardware accelerators.

AWS 123

AWS Machine Learning Machine Learning ML 123

AWS Machine Learning Blog

OCTOBER 5, 2023

In this post, we walk through how to fine-tune Llama 2 on AWS Trainium , a purpose-built accelerator for LLM training, to reduce training times and costs. We review the fine-tuning scripts provided by the AWS Neuron SDK (using NeMo Megatron-LM), the various configurations we used, and the throughput results we saw.

AWS 124

AWS Machine Learning Machine Learning Deep Learning 124

AWS Machine Learning Blog

MAY 30, 2024

CBRE is unlocking the potential of artificial intelligence (AI) to realize value across the entire commercial real estate lifecycle—from guiding investment decisions to managing buildings. The opportunities to unlock value using AI in the commercial real estate lifecycle starts with data at scale.

AWS 127

AWS SQL Database AI 127

AWS Machine Learning Blog

APRIL 4, 2023

AWS provides various services catered to time series data that are low code/no code, which both machine learning (ML) and non-ML practitioners can use for building ML solutions. In this post, we seek to separate a time series dataset into individual clusters that exhibit a higher degree of similarity between its data points and reduce noise.

Clustering 101

Clustering ML ML AWS 101

AWS Machine Learning Blog

MAY 23, 2023

Just recently, generative AI applications have captured everyone’s attention and imagination. We are truly at an exciting inflection point in the widespread adoption of ML, and we believe every customer experience and application will be reinvented with generative AI.

AWS 107

AWS AI AI ML 107

AWS Machine Learning Blog

MARCH 15, 2024

Therefore, ML creates challenges for AWS customers who need to ensure privacy and security across distributed entities without compromising patient outcomes. Solution overview We deploy FedML into multiple EKS clusters integrated with SageMaker for experiment tracking. As always, AWS welcomes your feedback.

AWS 139

AWS ML ML Machine Learning 139

AWS Machine Learning Blog

MAY 31, 2023

With containers, scaling on a cluster becomes much easier. In late 2022, AWS announced the general availability of Amazon EC2 Trn1 instances powered by AWS Trainium accelerators, which are purpose built for high-performance deep learning training. What you get is an ML development environment that is consistent and portable.

AWS 111

AWS Machine Learning Machine Learning ML 111

AWS Machine Learning Blog

APRIL 29, 2024

Close collaboration with AWS Trainium has also played a major role in making the Arcee platform extremely performant, not only accelerating model training but also reducing overall costs and enforcing compliance and data integrity in the secure AWS environment. Our cluster consisted of 16 nodes, each equipped with a trn1n.32xlarge

AWS 121

AWS Clustering ML ML 121

AWS Machine Learning Blog

JUNE 11, 2024

Each of these products are infused with artificial intelligence (AI) capabilities to deliver exceptional customer experience. Sprinklr’s specialized AI models streamline data processing, gather valuable insights, and enable workflows and analytics at scale to drive better decision-making and productivity.

Machine Learning 117

Machine Learning Machine Learning AWS Natural Language Processing 117

AWS Machine Learning Blog

DECEMBER 12, 2023

In this post, we’ll summarize training procedure of GPT NeoX on AWS Trainium , a purpose-built machine learning (ML) accelerator optimized for deep learning training. M tokens/$) trained such models with AWS Trainium without losing any model quality. We’ll outline how we cost-effectively (3.2 billion in Pythia. billion in Pythia.

AWS 123

AWS Machine Learning Machine Learning Deep Learning 123

Smart Data Collective

FEBRUARY 20, 2022

AWS (Amazon Web Services), the comprehensive and evolving cloud computing platform provided by Amazon, is comprised of infrastructure as a service (IaaS), platform as a service (PaaS) and packaged software as a service (SaaS). With its wide array of tools and convenience, AWS has already become a popular choice for many SaaS companies.

AWS 119

AWS Cloud Computing Data Lakes Database 119

AWS Machine Learning Blog

MARCH 15, 2023

Tens of thousands of AWS customers use AWS machine learning (ML) services to accelerate their ML development with fully managed infrastructure and tools. The data scientist is responsible for moving the code into SageMaker, either manually or by cloning it from a code repository such as AWS CodeCommit.

AWS 144

AWS Machine Learning Machine Learning Data Scientist 144

IBM Journey to AI blog

OCTOBER 26, 2023

Maximo maintains high-value assets with AI and analytics to optimize performance, extend asset lifecycles and reduce downtime and costs. In this journey, we are seeing an increased interest in migrating and deploying MAS on AWS Cloud. Therefore, Maximo migration and modernization has become even more important for clients.

AWS 116

AWS Clustering Database Analytics 116

Data Science Dojo

FEBRUARY 20, 2023

It is used by businesses across industries for a wide range of applications, including fraud prevention, marketing automation, customer service, artificial intelligence (AI), chatbots, virtual assistants, and recommendations. It provides a large cluster of clusters on a single machine.

Machine Learning 195

Machine Learning Machine Learning AWS Azure 195

AWS Machine Learning Blog

APRIL 19, 2024

Iambic Therapeutics is a drug discovery startup with a mission to create innovative AI-driven technologies to bring better medicines to cancer patients, faster. Our advanced generative and predictive artificial intelligence (AI) tools enable us to search the vast space of possible drug molecules faster and more effectively.

Clustering 123

Clustering AI AI AWS 123

AWS Machine Learning Blog

NOVEMBER 13, 2024

In the context of generative AI , significant progress has been made in developing multimodal embedding models that can embed various data modalities—such as text, image, video, and audio data—into a shared vector space. The AWS Command Line Interface (AWS CLI) installed on your machine to upload the dataset to Amazon S3.

AWS 114

AWS Database K-nearest Neighbors AI 114

AWS Machine Learning Blog

JANUARY 15, 2025

This post introduces HCLTechs AutoWise Companion, a transformative generative AI solution designed to enhance customers vehicle purchasing journey. Powered by generative AI services on AWS and large language models (LLMs) multi-modal capabilities, HCLTechs AutoWise Companion provides a seamless and impactful experience.

AWS 103

AWS SQL AI AI 103

AWS Machine Learning Blog

NOVEMBER 30, 2023

Building foundation models (FMs) requires building, maintaining, and optimizing large clusters to train models with tens to hundreds of billions of parameters on vast amounts of data. Customers such as Stability AI use SageMaker HyperPod to train their foundation models, including Stable Diffusion. “As

Clustering 120

Clustering AWS Machine Learning Machine Learning 120

Flipboard

NOVEMBER 17, 2023

Generative AI models have the potential to revolutionize enterprise operations, but businesses must carefully consider how to harness their power while overcoming challenges such as safeguarding data and ensuring the quality of AI-generated content. Delete the MongoDB Atlas cluster. Set up the database access and network access.

K-nearest Neighbors 170

K-nearest Neighbors AWS Clustering Database 170

AWS Machine Learning Blog

OCTOBER 2, 2024

Large enterprises are building strategies to harness the power of generative AI across their organizations. Managing bias, intellectual property, prompt safety, and data integrity are critical considerations when deploying generative AI solutions at scale. We focus on the operational excellence pillar in this post.

AWS 127

AWS AI AI ML 127

AWS Machine Learning Blog

APRIL 1, 2024

Distributed model training requires a cluster of worker nodes that can scale. Amazon Elastic Kubernetes Service (Amazon EKS) is a popular Kubernetes-conformant service that greatly simplifies the process of running AI/ML workloads, making it more manageable and less time-consuming.

Clustering 127

Clustering AWS ML ML 127

Flipboard

FEBRUARY 16, 2023

Modern model pre-training often calls for larger cluster deployment to reduce time and cost. In October 2022, we launched Amazon EC2 Trn1 Instances , powered by AWS Trainium , which is the second generation machine learning accelerator designed by AWS. The following diagram shows an example.

Clustering 132

Clustering AWS Deep Learning Deep Learning 132

AWS Machine Learning Blog

MAY 31, 2024

In this blog post and open source project , we show you how you can pre-train a genomics language model, HyenaDNA , using your genomic data in the AWS Cloud. Amazon SageMaker Amazon SageMaker is a fully managed ML service offered by AWS, designed to reduce the time and cost associated with training and tuning ML models at scale.

AWS 115

AWS ML ML Machine Learning 115

AWS Machine Learning Blog

AUGUST 21, 2023

Data is frequently kept in data lakes that can be managed by AWS Lake Formation , giving you the ability to implement fine-grained access control using a straightforward grant or revoke procedure. You can use the provided AWS CloudFormation stack to set up the architectural components for this solution.

AWS 98

AWS Data Lakes Clustering Data Preparation 98

IBM Journey to AI blog

JANUARY 16, 2024

Businesses are increasingly embracing data-intensive workloads, including high-performance computing, artificial intelligence (AI) and machine learning (ML). To aid in building more sustainable IT estates, IBM has partnered up with Amazon Web Services (AWS) to facilitate sustainable cloud modernization journeys.

AWS 109

AWS Clustering Database Artificial Intelligence 109

IBM Journey to AI blog

AUGUST 30, 2023

IBM Maximo Application Suite (MAS), the next generation of IBM Maximo, delivers a better user experience, faster integration, robust AI analytics and a broad range of cloud deployment options. Why ROSA for Maximo in AWS? Red Hat OpenShift Service on AWS is jointly developed and jointly supported by AWS and Red Hat.

AWS 84

AWS Clustering Internet of Things Artificial Intelligence 84

AWS Machine Learning Blog

SEPTEMBER 3, 2024

From deriving insights to powering generative artificial intelligence (AI) -driven applications, the ability to efficiently process and analyze large datasets is a vital capability. SageMaker Studio runs inside an AWS managed virtual private cloud ( VPC ), with network access for SageMaker Studio domains, in this setup configured as VPC-only.

AWS 116

AWS Clustering Big Data Big Data 116

AWS Machine Learning Blog

JULY 17, 2023

As described in the AWS Well-Architected Framework , separating workloads across accounts enables your organization to set common guardrails while isolating environments. Organizations with a multi-account architecture typically have Amazon Redshift and SageMaker Studio in two separate AWS accounts. Select VPC Only , then choose Next.

Clustering 96

Clustering AWS ML ML 96

AWS Machine Learning Blog

APRIL 24, 2024

Conversational artificial intelligence (AI) assistants are engineered to provide precise, real-time responses through intelligent routing of queries to the most suitable AI functions. With AWS generative AI services like Amazon Bedrock , developers can create systems that expertly manage and respond to user requests.

AWS 136

AWS AI AI SQL 136