article thumbnail

Scale ML workflows with Amazon SageMaker Studio and Amazon SageMaker HyperPod

AWS Machine Learning Blog

Solution overview Implementing the solution consists of the following high-level steps: Set up your environment and the permissions to access Amazon HyperPod clusters in SageMaker Studio. You can now use SageMaker Studio to discover the SageMaker HyperPod clusters, and view cluster details and metrics.

ML 101
article thumbnail

Integrate HyperPod clusters with Active Directory for seamless multi-user login

AWS Machine Learning Blog

Amazon SageMaker HyperPod is purpose-built to accelerate foundation model (FM) training, removing the undifferentiated heavy lifting involved in managing and optimizing a large training compute cluster. In this solution, HyperPod cluster instances use the LDAPS protocol to connect to the AWS Managed Microsoft AD via an NLB.

professionals

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

article thumbnail

The ultimate guide to Hyper-V backups for VMware administrators

Data Science Dojo

From vCenter, administrators can configure and control ESXi hosts, datacenters, clusters, traditional storage, software-defined storage, traditional networking, software-defined networking, and all other aspects of the vSphere architecture. VMware “clustering” is purely for virtualization purposes.

article thumbnail

Configure cross-account access of Amazon Redshift clusters in Amazon SageMaker Studio using VPC peering

AWS Machine Learning Blog

In this post, we walk through step-by-step instructions to establish a cross-account connection to any Amazon Redshift node type (RA3, DC2, DS2) by connecting the Amazon Redshift cluster located in one AWS account to SageMaker Studio in another AWS account in the same Region using VPC peering.

article thumbnail

Multi-account support for Amazon SageMaker HyperPod task governance

AWS Machine Learning Blog

In this post, we discuss how an enterprise with multiple accounts can access a shared Amazon SageMaker HyperPod cluster for running their heterogenous workloads. Account A hosts the SageMaker HyperPod cluster. To access Account A’s EKS cluster as a user in Account B, you will need to assume a cluster access role in Account A.

article thumbnail

Use LangChain with PySpark to process documents at massive scale with Amazon SageMaker Studio and Amazon EMR Serverless

AWS Machine Learning Blog

Cost optimization – The serverless nature of the integration means you only pay for the compute resources you use, rather than having to provision and maintain a persistent cluster. This same interface is also used for provisioning EMR clusters. The following diagram illustrates this solution.

AWS 119
article thumbnail

Using Geographic Data To Create A Perfect Google Maps Radius

Smart Data Collective

In 2012, Google boasted about its capabilities of using big data to create storytelling via interactive maps. Grouping radii can serve to visually demonstrate a cluster of comparative data within a particular location. The highly intuitive data interface provided by Google Maps can be very helpful.

Big Data 134