IBM Journey to AI blog

JANUARY 16, 2024

To aid in building more sustainable IT estates, IBM has partnered up with Amazon Web Services (AWS) to facilitate sustainable cloud modernization journeys. To read about other key scenarios and entry points of IBM Consulting® Custom Lens for Sustainability, check out the blog post: Sustainable App Modernization Using AWS Cloud.

AWS 109

AWS Clustering Database Artificial Intelligence 109

AWS Machine Learning Blog

APRIL 19, 2024

The architecture deploys a simple service in a Kubernetes pod within an EKS cluster. The Kubernetes Event Driven Autoscaler ( KEDA ) is configured to automatically scale the number of service pods, based on the custom metrics available in Prometheus. The service is exposed behind a reverse-proxy using Traefik.

Clustering 132

Clustering AI AI AWS 132

IBM Journey to AI blog

MARCH 20, 2024

Perhaps you need to discover what’s happening in your business and respond quickly to events. In the rest of this paper, we will explore how Cloud Pak for Integration, deployed on Red Hat OpenShift is the best way to provide integrations deploying in AWS. ROSA is jointly engineered and supported by AWS and Red Hat.

AWS 78

AWS Clustering AI AI 78

AWS Machine Learning Blog

JANUARY 17, 2024

It can represent a geographical area as a whole or it can represent an event associated with a geographical area. We then discuss the various use cases and explore how you can use AWS services to clean the data, how machine learning (ML) can aid in this effort, and how you can make ethical use of the data in generating visuals and insights.

Clustering 135

Clustering AWS ML ML 135

AWS Machine Learning Blog

JULY 17, 2023

As described in the AWS Well-Architected Framework , separating workloads across accounts enables your organization to set common guardrails while isolating environments. Organizations with a multi-account architecture typically have Amazon Redshift and SageMaker Studio in two separate AWS accounts.

Clustering 88

Clustering AWS ML ML 88

AWS Machine Learning Blog

OCTOBER 2, 2024

In this post, we share AWS guidance that we have learned and developed as part of real-world projects into practical guides oriented towards the AWS Well-Architected Framework , which is used to build production infrastructure and applications on AWS. We focus on the operational excellence pillar in this post.

AWS 131

AWS AI AI ML 131

AWS Machine Learning Blog

APRIL 18, 2024

It’s straightforward to deploy in your AWS account. Prerequisites You need to have an AWS account and an AWS Identity and Access Management (IAM) role and user with permissions to create and manage the necessary resources and components for this application. Everything you need is provided as open source in our GitHub repo.

AWS 138

AWS Analytics Analytics AI 138

AWS Machine Learning Blog

MAY 1, 2024

Virginia) AWS Region. The diagram details a comprehensive AWS Cloud-based setup within a specific Region, using multiple AWS services. The primary interface for the chatbot is a Streamlit application hosted on an Amazon Elastic Container Service (Amazon ECS) cluster, with accessibility managed by an Application Load Balancer.

AWS 134

AWS Machine Learning Machine Learning SQL 134

AWS Machine Learning Blog

MAY 16, 2024

The main AWS services used are SageMaker, Amazon EMR , AWS CodeBuild , Amazon Simple Storage Service (Amazon S3), Amazon EventBridge , AWS Lambda , and Amazon API Gateway. The subsequent role of EventBridge was to dispatch events, instigated by the alteration of the buildspec.yml file, leading to running CodeBuild.

AWS 130

AWS ML ML Deep Learning 130

Flipboard

NOVEMBER 27, 2024

In the context of enterprise data asset search powered by a metadata catalog hosted on services such Amazon DataZone, AWS Glue, and other third-party catalogs, knowledge graphs can help integrate this linked data and also enable a scalable search paradigm that integrates metadata that evolves over time.

AWS 149

AWS Database ML ML 149

AWS Machine Learning Blog

AUGUST 8, 2023

You can run Spark applications interactively from Amazon SageMaker Studio by connecting SageMaker Studio notebooks and AWS Glue Interactive Sessions to run Spark jobs with a serverless cluster. With interactive sessions, you can choose Apache Spark or Ray to easily process large datasets, without worrying about cluster management.

AWS 88

AWS Clustering Machine Learning Machine Learning 88

AWS Machine Learning Blog

DECEMBER 18, 2023

Make sure you have the following prerequisites: Create an S3 bucket Configure MongoDB Atlas cluster Create a free MongoDB Atlas cluster by following the instructions in Create a Cluster. In his role Igor is working with strategic partners helping them build complex, AWS-optimized architectures. Note we have two folders.

Clustering 131

Clustering AWS Database ML 131

AWS Machine Learning Blog

JANUARY 13, 2023

Analyzing real-world healthcare and life sciences (HCLS) data poses several practical challenges, such as distributed data silos, lack of sufficient data at a single site for rare events, regulatory guidelines that prohibit data sharing, infrastructure requirement, and cost incurred in creating a centralized data repository.

AWS 97

AWS Analytics Analytics Machine Learning 97

AWS Machine Learning Blog

SEPTEMBER 12, 2024

In this post, we explore the journey that Thomson Reuters took to enable cutting-edge research in training domain-adapted large language models (LLMs) using Amazon SageMaker HyperPod , an Amazon Web Services (AWS) feature focused on providing purpose-built infrastructure for distributed training at scale. So, for example, a 6.6B

Clustering 110

Clustering AWS ML ML 110

Smart Data Collective

AUGUST 17, 2022

You can safely use an Apache Kafka cluster for seamless data movement from the on-premise hardware solution to the data lake using various cloud services like Amazon’s S3 and others. Amazon’s AWS Glue is one such tool that allows you to consume data from Apache Kafka and Amazon-managed streaming for Apache Kafka (MSK).

Apache Kafka 122

Apache Kafka ETL Data Lakes Clustering 122

AWS Machine Learning Blog

MARCH 6, 2023

These factors require training an LLM over large clusters of accelerated machine learning (ML) instances. In the past few years, numerous customers have been using the AWS Cloud for LLM training. We recommend working with your AWS account team or contacting AWS Sales to determine the appropriate Region for your LLM workload.

AWS 111

AWS Clustering ML ML 111

AWS Machine Learning Blog

NOVEMBER 3, 2023

This process comprises two key components: event data and optical tracking data. Event data collection entails gathering the fundamental building blocks of the game. For the precision needed in shot speed calculations, we must ensure that the ball’s position aligns precisely with the moment of the event.

AWS 127

AWS Apache Kafka Data Scientist Data Science 127

AWS Machine Learning Blog

APRIL 25, 2024

We provide a comprehensive guide on how to deploy speaker segmentation and clustering solutions using SageMaker on the AWS Cloud. Solution overview Amazon Transcribe is the go-to service for speaker diarization in AWS. Make sure the AWS account has a service quota for hosting a SageMaker endpoint for an ml.g5.2xlarge instance.

AWS 129

AWS ML ML Python 129

AWS Machine Learning Blog

MAY 3, 2023

AWS customer Vericast is a marketing solutions company that makes data-driven decisions to boost marketing ROIs for its clients. Dynamic scaling of feature engineering jobs – A combination of various AWS services is used for this, but most notably SageMaker Processing.

AWS 103

AWS Machine Learning Machine Learning ML 103

Data Science Dojo

JULY 24, 2023

Clusters : Clusters are groups of interconnected nodes that work together to process and store data. Clustering allows for improved performance and fault tolerance as tasks can be distributed across nodes. Amazon S3: Amazon Simple Storage Service (S3) is a scalable object storage service provided by Amazon Web Services (AWS).

Big Data 195

Big Data Big Data Data Engineering Data Engineer 195

IBM Journey to AI blog

FEBRUARY 15, 2024

Logs Logs include discrete events recorded every time something occurs in the system, such as status or error messages, or transaction details. Autoscaling When traffic spikes, Kubernetes can automatically spin up new clusters to handle the additional workload. Kubernetes logs can be written in both structured and unstructured text.

Clustering 84

Clustering Azure Data Visualization AWS 84

AWS Machine Learning Blog

NOVEMBER 18, 2024

To address this challenge, this post demonstrates a proactive approach for security vulnerability assessment of your accounts and workloads, using Amazon GuardDuty , Amazon Bedrock , and other AWS serverless technologies. This form of automation can help improve efficiency and reduce the response time to security threats.

AWS 94

AWS ML ML AI 94

Towards AI

AUGUST 17, 2023

Each environment has a dedicated AWS account with its own cluster and ArgoCD installation. Potential Solutions At the outset, two potential solutions were proposed: ECR Cross Account Replication: AWS’s ECR natively supports replicating images between two accounts. Source: Image by the author. hook: PreSync argocd.argoproj.io/hook-delete-policy:

AWS 98

AWS Clustering AI AI 98

AWS Machine Learning Blog

DECEMBER 15, 2023

Prerequisites To implement this solution, complete the following prerequisites: Have AWS Cloud admin access with an AWS Identity and Access Management (IAM) user with permissions required to complete the integration. Enter a connection name such as demo and choose your desired Amazon DocumentDB cluster. Choose Add connection.

Machine Learning 126

Machine Learning Machine Learning AWS ML 126

AWS Machine Learning Blog

SEPTEMBER 11, 2024

The following figure illustrates the idea of a large cluster of GPUs being used for learning, followed by a smaller number for inference. Examples of other PBAs now available include AWS Inferentia and AWS Trainium , Google TPU, and Graphcore IPU. Suppliers of data center GPUs include NVIDIA, AMD, Intel, and others.

AWS 113

AWS ML ML Clustering 113

AWS Machine Learning Blog

FEBRUARY 24, 2023

AWS recently released Amazon SageMaker geospatial capabilities to provide you with satellite imagery and geospatial state-of-the-art machine learning (ML) models, reducing barriers for these types of use cases. Given the highly parallel needs, we chose Lambda to process our images.

ML 104

ML ML AWS Data Pipeline 104

AWS Machine Learning Blog

NOVEMBER 16, 2023

This is a joint blog with AWS and Philips. Since 2014, the company has been offering customers its Philips HealthSuite Platform, which orchestrates dozens of AWS services that healthcare and life sciences companies use to improve patient care.

ML 125

ML ML AWS AI 125

AWS Machine Learning Blog

SEPTEMBER 18, 2023

With Ray and AIR, the same Python code can scale seamlessly from a laptop to a large cluster. The full code can be found on the aws-samples-for-ray GitHub repository. Prepare the source data for the feature store by adding an event time and record ID for each row of data. You can specify resource requirements in actors too.

Machine Learning 134

Machine Learning Machine Learning ML ML 134

AWS Machine Learning Blog

JANUARY 6, 2023

Many AWS media and entertainment customers license IMDb data through AWS Data Exchange to improve content discovery and increase customer engagement and retention. We downloaded the data from AWS Data Exchange and processed it in AWS Glue to generate KG files. Background. That’s an out-of-catalog search experience!

AWS 104

AWS ML ML Machine Learning 104

IBM Journey to AI blog

OCTOBER 17, 2023

The key components of Instana are host agents and agent sensors deployed on platforms like IBM Cloud®, AWS, and Azure. Supported cloud platforms with IBM Instana IBM Instana supports IBM Cloud, AWS, Azure and SAP. Currently, Instana supports SAP BTP Kyma cluster monitoring.

Clustering 92

Clustering Azure AWS Database 92

AWS Machine Learning Blog

JUNE 7, 2023

SageMaker Training provisions compute clusters with user-defined hardware configuration and code. TII used transient clusters provided by the SageMaker Training API to train the Falcon LLM, up to 48 ml.p4d.24xlarge Checkpoints also need to reach the training cluster as fast as possible in the event of job restart.

Clustering 105

Clustering Machine Learning Machine Learning AWS 105

AWS Machine Learning Blog

AUGUST 24, 2023

Today, we’re pleased to announce the preview of Amazon SageMaker Profiler , a capability of Amazon SageMaker that provides a detailed view into the AWS compute resources provisioned during training deep learning models on SageMaker. The following table provides the links to the supported AWS Deep Learning Containers for SageMaker.

AWS 96

AWS Deep Learning Deep Learning ML 96

Flipboard

MARCH 13, 2023

Therefore, VMware Carbon Black and AWS chose to build a custom MLOps pipeline using Amazon SageMaker for its ease of use, versatility, and fully managed infrastructure. In this post, VMware Carbon Black and AWS architects discuss how we built and managed custom ML workflows using Gitlab , Amazon MWAA, and SageMaker.

ML 123

ML ML AWS Data Scientist 123

Data Science Dojo

JULY 3, 2024

Machine Learning : Supervised and unsupervised learning algorithms, including regression, classification, clustering, and deep learning. Big Data Technologies : Handling and processing large datasets using tools like Hadoop, Spark, and cloud platforms such as AWS and Google Cloud.

Data Science 150

Data Science Machine Learning Machine Learning Data Visualization 150

AWS Machine Learning Blog

SEPTEMBER 19, 2023

Prerequisites To follow this tutorial, you need the following: An AWS account. AWS Identity and Access Management (IAM) permissions. We add the record identifier model_year_status and event time ingest_time to this feature group. Add the event time to the ingest_time column. Use aggregated features to train an ML model.

ML 129

ML ML AWS SQL 129

AWS Machine Learning Blog

MAY 10, 2023

The SageMaker extension expects the JupyterLab environment to have valid AWS credentials and permissions to schedule notebook jobs. We discuss the steps for setting up credentials and AWS Identity and Access Management (IAM) permissions later in this post. See Installing or updating the latest version of the AWS CLI for instructions.

AWS 100

AWS Data Scientist ML ML 100

IBM Journey to AI blog

NOVEMBER 13, 2023

Nodes run the pods and are usually grouped in a Kubernetes cluster, abstracting the underlying physical hardware resources. As an open-source system, Kubernetes services are supported by all the leading public cloud providers, including IBM, Amazon Web Services (AWS), Microsoft Azure and Google.

Machine Learning 129

Machine Learning Machine Learning ML ML 129