AWS Machine Learning Blog

SEPTEMBER 18, 2023

Machine learning (ML) is becoming increasingly complex as customers try to solve more and more challenging problems. This complexity often leads to the need for distributed ML, where multiple machines are used to train a single model. SageMaker is a fully managed service for building, training, and deploying ML models.

Machine Learning 127

Machine Learning Machine Learning ML ML 127

The MLOps Blog

MARCH 20, 2023

Long-term ML project involves developing and sustaining applications or systems that leverage machine learning models, algorithms, and techniques. An example of a long-term ML project will be a bank fraud detection system powered by ML models and algorithms for pattern recognition. 2 Ensuring and maintaining high-quality data.

ML 59

ML ML Machine Learning Machine Learning 59

The MLOps Blog

AUGUST 4, 2023

Situations described above arise way too often in ML teams, and their consequences vary from a single developer’s annoyance to the team’s inability to ship their code as needed. Let’s dive into the world of monorepos, an architecture widely adopted in major tech companies like Google, and how they can enhance your ML workflows.

ML 52

ML ML Machine Learning Machine Learning 52

ODSC - Open Data Science

FEBRUARY 2, 2023

To get a better grip on those changes we reviewed over 25,000 data scientist job descriptions from that past year to find out what employers are looking for in 2023. Much of what we found was to be expected, though there were definitely a few surprises. Employers aren’t just looking for people who can program.

Data Science 98

Data Science Data Scientist Computer Science Computer Science 98

Tableau

OCTOBER 8, 2021

With its LookML modeling language, Looker provides a unique, modern approach to define governed and reusable data models to build a trusted foundation for analytics. Connecting directly to this semantic layer will help give customers access to critical business data in a safe, governed manner. Direct connection to Google BigQuery.

Tableau 138

Tableau Analytics Analytics Machine Learning 138

Iguazio

JANUARY 22, 2024

Definitions: Foundation Models, Gen AI, and LLMs Before diving into the practice of productizing LLMs, let’s review the basic definitions of GenAI elements: Foundation Models (FMs) - Large deep learning models that are pre-trained with attention mechanisms on massive datasets. This helps cleanse the data.

Data Pipeline 52

Data Pipeline ML ML Data Warehouse 52

DagsHub

APRIL 7, 2024

Data scientists and machine learning engineers need to collaborate to make sure that together with the model, they develop robust data pipelines. These pipelines cover the entire lifecycle of an ML project, from data ingestion and preprocessing, to model training, evaluation, and deployment.

Machine Learning 59

Machine Learning Machine Learning ML ML 59

AWS Machine Learning Blog

SEPTEMBER 11, 2024

These activities cover disparate fields such as basic data processing, analytics, and machine learning (ML). ML is often associated with PBAs, so we start this post with an illustrative figure. The ML paradigm is learning followed by inference. The union of advances in hardware and ML has led us to the current day.

AWS 101

AWS ML ML Clustering 101

Tableau

OCTOBER 8, 2021

With its LookML modeling language, Looker provides a unique, modern approach to define governed and reusable data models to build a trusted foundation for analytics. Connecting directly to this semantic layer will help give customers access to critical business data in a safe, governed manner. Direct connection to Google BigQuery.

Tableau 98

Tableau Analytics Analytics Machine Learning 98

Alation

APRIL 14, 2022

And, as organizations progress and grow, “data drift” starts to impact data usage, models, and your business. In today’s AI/ML-driven world of data analytics, explainability needs a repository just as much as those doing the explaining need access to metadata, EG, information about the data being used.

Data Governance 59

Data Governance ML ML Cloud Data 59

DagsHub

OCTOBER 23, 2024

Managing unstructured data is essential for the success of machine learning (ML) projects. Without structure, data is difficult to analyze and extracting meaningful insights and patterns is challenging. This article will discuss managing unstructured data for AI and ML projects. What is Unstructured Data?

Machine Learning 52

Machine Learning Machine Learning Data Lakes AI 52

Snorkel AI

JUNE 22, 2023

One should really think of us at the level of doing the technical implementation work around designing, developing and operationally deploying data products and services that use ML. I’ll give you a rough guide to what we’ll talk about—in the first place, a very macro and micro view of the importance of data.

Data Quality 52

Data Quality ML ML AI 52

Snorkel AI

JUNE 22, 2023

One should really think of us at the level of doing the technical implementation work around designing, developing and operationally deploying data products and services that use ML. I’ll give you a rough guide to what we’ll talk about—in the first place, a very macro and micro view of the importance of data.

Data Quality 52

Data Quality ML ML AI 52

DagsHub

DECEMBER 5, 2023

This is where ML experiment tracking comes into play! What is ML Experiment Tracking? ML experiment tracking is the process of recording, organizing, and analyzing the results of ML experiments. It helps data scientists keep track of their experiments, reproduce their results, and collaborate with others effectively.

Machine Learning 52

Machine Learning Machine Learning ML ML 52

Snorkel AI

JUNE 22, 2023

One should really think of us at the level of doing the technical implementation work around designing, developing and operationally deploying data products and services that use ML. I’ll give you a rough guide to what we’ll talk about—in the first place, a very macro and micro view of the importance of data.

Data Quality 52

Data Quality ML ML AI 52

The MLOps Blog

MARCH 15, 2023

This includes the tools and techniques we used to streamline the ML model development and deployment processes, as well as the measures taken to monitor and maintain models in a production environment. Costs: Oftentimes, cost is the most important aspect of any ML model deployment. This includes data quality, privacy, and compliance.

AWS 52

AWS ETL ML ML 52

The MLOps Blog

MARCH 28, 2024

Strategies for improving GPU usage include mixed-precision training, optimizing data transfer and processing, and appropriately dividing workloads between CPU and GPU. GPU and CPU metrics can be monitored using an ML experiment tracker like Neptune, enabling teams to identify bottlenecks and systematically improve training performance.

Deep Learning 52

Deep Learning Deep Learning Data Pipeline Machine Learning 52

Mlearning.ai

JUNE 16, 2023

GPT-4 Data Pipelines: Transform JSON to SQL Schema Instantly Blockstream’s public Bitcoin API. The data would be interesting to analyze. From Data Engineering to Prompt Engineering Prompt to do data analysis BI report generation/data analysis In BI/data analysis world, people usually need to query data (small/large).

AI 52

AI AI Data Analysis Data Analysis 52

phData

AUGUST 15, 2024

In traditional machine learning , data pipelines feeding into the model have queries written with idempotency in mind, and data validation checks are performed before and after inference to confirm an expected output. If you missed the other blogs in the series, definitely check them out!

AI 52

AI AI Data Engineering Data Engineering 52

Snorkel AI

JULY 3, 2023

As the algorithms we use have gotten more robust and we have increased our compute power through new technologies, we haven’t made nearly as much progress on the data part of our jobs. Because of this, I’m always looking for ways to automate and improve our data pipelines. So why should we use data pipelines?

Exploratory Data Analysis 52

Exploratory Data Analysis Data Pipeline Data Scientist Machine Learning 52

Snorkel AI

JULY 3, 2023

As the algorithms we use have gotten more robust and we have increased our compute power through new technologies, we haven’t made nearly as much progress on the data part of our jobs. Because of this, I’m always looking for ways to automate and improve our data pipelines. So why should we use data pipelines?

Exploratory Data Analysis 52

Exploratory Data Analysis Data Pipeline Data Scientist Machine Learning 52

ODSC - Open Data Science

JANUARY 3, 2025

Fine-tune Your Own Open-Source SLMs Devvret Rishi, CEO of Predibase, and Chloe Leung, ML solutions architect at Predibase Discover how to cost-effectively customize open-source small language models (SLMs) to outperform GPT-4 on various tasks. Cloning NotebookLM with Open Weights Models Niels Bantilan, Chief ML Engineer atUnion.AI

AI 40

AI AI ML ML 40

Snorkel AI

MARCH 2, 2023

I am Ali Arsanjani, and I lead partner engineering for Google Cloud, specializing in the area of AI-ML, and I’m very happy to be here today with everyone. Then we’re going to talk about adapting foundation models for the enterprise and how that affects the ML lifecycle, and what we need to potentially add to the lifecycle.

Machine Learning 52

Machine Learning Machine Learning Data Preparation AI 52

Snorkel AI

MARCH 2, 2023

I am Ali Arsanjani, and I lead partner engineering for Google Cloud, specializing in the area of AI-ML, and I’m very happy to be here today with everyone. Then we’re going to talk about adapting foundation models for the enterprise and how that affects the ML lifecycle, and what we need to potentially add to the lifecycle.

Machine Learning 52

Machine Learning Machine Learning Data Preparation AI 52

phData

JULY 5, 2024

It is definitely an exciting time as the open-source community enhances and builds out these frameworks, but they are still being refined with best practices and new features. Conclusion This blog has only covered the minimum technologies required to build the bare bones of a generative AI application.

AI 52

AI AI Database AWS 52

Mlearning.ai

JULY 10, 2023

In case of complex data pipelines, a combination of Materialized Views, Stored Procedures, and Scheduled Queries could be a better choice than to solely rely on Scheduled Queries by itself.

SQL 52

SQL Database Database Administration Data Lakes 52

phData

JULY 18, 2023

Why Migrate to a Modern Data Stack? Data teams can focus on delivering higher-value data tasks with better organizational visibility. Move Beyond One-off Analytics: The Modern Data Stack empowers you to elevate your data for advanced analytics and integration of AI/ML, enabling faster generation of actionable business insights.

Data Warehouse 52

Data Warehouse Analytics Analytics SQL 52

Mlearning.ai

MARCH 15, 2023

I have checked the AWS S3 bucket and Snowflake tables for a couple of days and the Data pipeline is working as expected. The scope of this article is quite big, we will exercise the core steps of data science, let's get started… Project Layout Here are the high-level steps for this project.

Python 52

Python AWS Exploratory Data Analysis Machine Learning 52

phData

JUNE 7, 2023

Data scientists use data-driven approaches to enable AI systems to make better predictions, optimize decision-making, and uncover hidden patterns that ultimately drive innovation and improve performance across various domains. It includes techniques like supervised, unsupervised, and reinforcement learning.

AI 59

AI AI Machine Learning Machine Learning 59

The MLOps Blog

AUGUST 11, 2023

There comes a time when every ML practitioner realizes that training a model in Jupyter Notebook is just one small part of the entire project. Getting a workflow ready which takes your data from its raw form to predictions while maintaining responsiveness and flexibility is the real deal.

ML 52

ML ML Machine Learning Machine Learning 52

phData

SEPTEMBER 27, 2024

All this raw data goes into your persistent stage. Then, if you later refine your definition of what constitutes an “engaged” customer, having the raw data in persistent staging allows for easy reprocessing of historical data with the new logic. Your customer data game will never be the same.

Data Models 52

Data Models Data Modeling Apache Kafka Data Lakes 52

AWS Machine Learning Blog

MARCH 27, 2025

In this post, we explore how you can use Amazon Bedrock to generate high-quality categorical ground truth data, which is crucial for training machine learning (ML) models in a cost-sensitive environment. This use case, solvable through ML, can enable support teams to better understand customer needs and optimize response strategies.

AWS 108

AWS ETL ML ML 108

AWS Machine Learning Blog

OCTOBER 24, 2024

The agent can generate SQL queries using natural language questions using a database schema DDL (data definition language for SQL) and execute them against a database instance for the database tier. We use Amazon Bedrock Agents with two knowledge bases for this assistant.

AWS 96

AWS SQL Database AI 96