AWS Machine Learning Blog

OCTOBER 2, 2023

We show you how to use AWS IoT Greengrass to manage model inference at the edge and how to automate the process using AWS Step Functions and other AWS services. AWS IoT Greengrass is an Internet of Things (IoT) open-source edge runtime and cloud service that helps you build, deploy, and manage edge device software.

AWS 110

AWS ML ML Internet of Things 110

Flipboard

NOVEMBER 21, 2024

We use Amazon SageMaker Pipelines , which helps automate the different steps, including data preparation, fine-tuning, and creating the model. Prerequisites For this walkthrough, complete the following prerequisite steps: Set up an AWS account. Create a SageMaker Studio environment.

AWS 129

AWS ML ML Algorithm 129

Flipboard

AUGUST 17, 2023

Amazon Redshift uses SQL to analyze structured and semi-structured data across data warehouses, operational databases, and data lakes, using AWS-designed hardware and ML to deliver the best price-performance at any scale. To do this, we provide an AWS CloudFormation template to create a stack that contains the resources.

ML 123

ML ML AWS Data Warehouse 123

AWS Machine Learning Blog

APRIL 17, 2023

In other words, companies need to move from a model-centric approach to a data-centric approach.” – Andrew Ng A data-centric AI approach involves building AI systems with quality data involving data preparation and feature engineering. Custom transforms can be written as separate steps within Data Wrangler.

AWS 93

AWS ML ML Python 93

AWS Machine Learning Blog

SEPTEMBER 14, 2023

The complexity of developing a bespoke classification machine learning model varies depending on a variety of aspects such as data quality, algorithm, scalability, and domain knowledge, to mention a few. We will introduce a custom classifier training pipeline that can be deployed in your AWS account with few clicks.

AWS 116

AWS Machine Learning Machine Learning Data Scientist 116

AWS Machine Learning Blog

FEBRUARY 12, 2024

One of the several challenges faced was adapting the existing on-premises pipeline solution for use on AWS. The solution involved two key components: Modifying and extending existing code – The first part of our solution involved the modification and extension of our existing code to make it compatible with AWS infrastructure.

ML 128

ML ML AWS Machine Learning 128

AWS Machine Learning Blog

SEPTEMBER 18, 2023

Amazon SageMaker Pipelines allows orchestrating the end-to-end ML lifecycle from data preparation and training to model deployment as automated workflows. The full code can be found on the aws-samples-for-ray GitHub repository. Solution overview This post focuses on the benefits of using Ray and SageMaker together.

Machine Learning 119

Machine Learning Machine Learning ML ML 119

AWS Machine Learning Blog

SEPTEMBER 20, 2023

For example, you might have acquired a company that was already running on a different cloud provider, or you may have a workload that generates value from unique capabilities provided by AWS. We show how you can build and train an ML model in AWS and deploy the model in another platform.

ML 119

ML ML Azure AWS 119

AWS Machine Learning Blog

OCTOBER 8, 2024

SageMaker AutoMLV2 is part of the SageMaker Autopilot suite, which automates the end-to-end machine learning workflow from data preparation to model deployment. Data preparation The foundation of any machine learning project is data preparation.

Machine Learning 90

Machine Learning Machine Learning Data Preparation AWS 90

AWS Machine Learning Blog

NOVEMBER 1, 2024

We discuss the important components of fine-tuning, including use case definition, data preparation, model customization, and performance evaluation. This post dives deep into key aspects such as hyperparameter optimization, data cleaning techniques, and the effectiveness of fine-tuning compared to base models.

Data Preparation 91

Data Preparation Machine Learning Machine Learning AI 91

AWS Machine Learning Blog

SEPTEMBER 11, 2024

Examples of other PBAs now available include AWS Inferentia and AWS Trainium , Google TPU, and Graphcore IPU. Around this time, industry observers reported NVIDIA’s strategy pivoting from its traditional gaming and graphics focus to moving into scientific computing and data analytics.

AWS 95

AWS ML ML Clustering 95

AWS Machine Learning Blog

AUGUST 22, 2023

With SageMaker, data scientists and developers can quickly build and train ML models, and then deploy them into a production-ready hosted environment. In this post, we demonstrate how to use the managed ML platform to provide a notebook experience environment and perform federated learning across AWS accounts, using SageMaker training jobs.

Machine Learning 77

Machine Learning Machine Learning AWS ML 77

AWS Machine Learning Blog

MAY 29, 2024

Figure 1: LLaVA architecture Prepare data When it comes to fine-tuning the LLaVA model for specific tasks or domains, data preparation is of paramount importance because having high-quality, comprehensive annotations enables the model to learn rich representations and achieve human-level performance on complex visual reasoning challenges.

ML 121

ML ML AWS Data Visualization 121

AWS Machine Learning Blog

NOVEMBER 15, 2023

Prerequisites The following are prerequisites for completing the walkthrough in this post: An AWS account Familiarity with SageMaker concepts, such as an Estimator, training job, and HPO job Familiarity with the Amazon SageMaker Python SDK Python programming knowledge Implement the solution The full code is available in the GitHub repo.

Algorithm 107

Algorithm AWS ML ML 107

Towards AI

AUGUST 16, 2023

Therefore, a common mistake when interviewing applicants is to focus on the minutia of a particular platform (AWS, GCP, Databricks, MLflow, etc.). A better definition would make use of the directed acyclic graph (DAG) since it may not be a linear process.

Machine Learning 88

Machine Learning Machine Learning ML ML 88

AWS Machine Learning Blog

JULY 5, 2023

Solution overview The AWS predictive maintenance solution for automotive fleets applies deep learning techniques to common areas that drive vehicle failures, unplanned downtime, and repair costs. The connected vehicle sends sensor logs to AWS IoT Core (alternatively, via an HTTP interface). Finally, you launch SageMaker Studio.

AWS 75

AWS Deep Learning Deep Learning ML 75

AWS Machine Learning Blog

AUGUST 3, 2023

SageMaker is a fully managed platform that enables developers and data scientists to build, train, and deploy ML models quickly, while also offering the cost-saving benefits of using the AWS Cloud infrastructure. These checkpoints can be used to resume training at a later moment or as a model to deploy on an endpoint.

ML 84

ML ML AWS Python 84

AWS Machine Learning Blog

SEPTEMBER 1, 2023

Generative AI definitions and differences to MLOps In classic ML, the preceding combination of people, processes, and technology can help you productize your ML use cases. The following is an example of notable proprietary FMs available in AWS (July 2023). Only prompt engineering is necessary for better results.

AI 123

AI AI ML ML 123

AWS Machine Learning Blog

FEBRUARY 27, 2023

Amazon SageMaker Clarify can detect potential bias during data preparation, after model training, and in your deployed model. The definition of these hyperparameters and others available with SageMaker AMT can be found here. About the authors Munish Dabra is a Senior Solutions Architect at Amazon Web Services (AWS).

ML 85

ML ML AWS Machine Learning 85

DagsHub

JULY 25, 2024

Launched by Microsoft, Azure ML provides a comprehensive suite of tools and services to support the entire machine learning lifecycle, from data preparation to model deployment and management. Further Reading and Documentation H2O.ai Documentation H2O.ai

Deep Learning 52

Deep Learning Deep Learning Machine Learning Machine Learning 52

Pickl AI

JULY 12, 2024

This section delves into its foundational definitions, types, and critical concepts crucial for comprehending its vast landscape. Data Preparation for AI Projects Data preparation is critical in any AI project, laying the foundation for accurate and reliable model outcomes.

Artificial Intelligence 52

Artificial Intelligence Artificial Intelligence Python Natural Language Processing 52

ODSC - Open Data Science

JANUARY 29, 2024

We don’t claim this is a definitive analysis but rather a rough guide due to several factors: Job descriptions show lagging indicators of in-demand prompt engineering skills, especially when viewed over the course of 9 months. The definition of a particular job role is constantly in flux and varies from employer to employer.

Data Science 52

Data Science Machine Learning Machine Learning Natural Language Processing 52

AWS Machine Learning Blog

JULY 13, 2023

We use HyperbandStrategyConfig to configure StrategyConfig , which is later used by the tuning job definition. In his spare time, he enjoys cycling, hiking, and complaining about data preparation. Based out of Israel, Uri works to empower enterprise customers to design, build, and operate ML workloads at scale.

Clustering 73

Clustering Algorithm Deep Learning Deep Learning 73

Pickl AI

NOVEMBER 18, 2024

Key steps involve problem definition, data preparation, and algorithm selection. Data quality significantly impacts model performance. Cloud Platforms for Machine Learning Cloud platforms like AWS, Google Cloud, and Microsoft Azure provide powerful infrastructures for building and deploying Machine Learning Models.

Machine Learning 52

Machine Learning Machine Learning Decision Trees Algorithm 52

AWS Machine Learning Blog

JANUARY 26, 2024

We also discuss common security concerns that can undermine trust in AI, as identified by the Open Worldwide Application Security Project (OWASP) Top 10 for LLM Applications , and show ways you can use AWS to increase your security posture and confidence while innovating with generative AI.

AWS 137

AWS ML ML AI 137

AWS Machine Learning Blog

SEPTEMBER 18, 2024

In addition to its groundbreaking AI innovations, Zeta Global has harnessed Amazon Elastic Container Service (Amazon ECS) with AWS Fargate to deploy a multitude of smaller models efficiently. It simplifies feature access for model training and inference, significantly reducing the time and complexity involved in managing data pipelines.

AWS 113

AWS Machine Learning Machine Learning ML 113

AWS Machine Learning Blog

NOVEMBER 30, 2023

The number of companies launching generative AI applications on AWS is substantial and building quickly, including adidas, Booking.com, Bridgewater Associates, Clariant, Cox Automotive, GoDaddy, and LexisNexis Legal & Professional, to name just a few. Innovative startups like Perplexity AI are going all in on AWS for generative AI.

AWS 133

AWS AI AI ML 133

AWS Machine Learning Blog

NOVEMBER 29, 2023

The following figure shows the framework to evaluate LLMs and LLM-based services: Amazon SageMaker Clarify LLM evaluation is an open-source Foundation Model Evaluation (FMEval) library developed by AWS to help customers easily evaluate LLMs. Jagdeep Singh Soni is a Senior Partner Solutions Architect at AWS based in Netherlands.

Algorithm 102

Algorithm ML ML Data Scientist 102

AWS Machine Learning Blog

SEPTEMBER 12, 2023

Solution overview Scalable Capital’s ML infrastructure consists of two AWS accounts: one as an environment for the development stage and the other one for the production stage. The following diagram shows the workflow for our email classifier project, but can also be generalized to other data science projects. Use Version 2.x

Data Science 115

Data Science Data Scientist AWS ML 115