Hacker News

OCTOBER 10, 2024

A new search engine in 2024? Yes, but stateless — index on S3, serverless — no cluster state, with all Lucene features — filters, autocomplete, facets. And also with local embedding & RAG inference.

Clustering 160

Clustering 160

Towards AI

MAY 8, 2024

Last Updated on May 9, 2024 by Editorial Team Author(s): Francis Adrian Viernes Originally published on Towards AI. K-means is probably one of the most clustering algorithms out there. It likewise provides an opportunity for customization to fit the unique setup of datasets, including the addition of conditionals.

Clustering 97

Clustering Algorithm Python AI 97

Towards AI

SEPTEMBER 3, 2024

Last Updated on September 3, 2024 by Editorial Team Author(s): Surya Maddula Originally published on Towards AI. We will discuss KNNs, also known as K-Nearest Neighbours and K-Means Clustering. This member-only story is on us. Upgrade to access all of Medium. Let’s discuss two popular ML algorithms, KNNs and K-Means.

K-nearest Neighbors 111

K-nearest Neighbors Clustering Supervised Learning ML 111

Hacker News

MARCH 12, 2024

Marking a major investment in Meta’s AI future, we are announcing two 24k GPU clusters. We use this cluster design for Llama 3 training. We built these clusters on top of Grand Teton , OpenRack , and PyTorch and continue to push open innovation across the industry. We are strongly committed to open compute and open source.

Clustering 182

Clustering AI AI ML 182

Towards AI

JUNE 3, 2024

Last Updated on June 4, 2024 by Editorial Team Author(s): Greg Postalian-Yrausquin Originally published on Towards AI. In my experience clustering sometimes works better working with principal components than with the actual values). Clustering")ax1.set_xlabel("Silhouette set_ylabel("Cluster labels")ax1.axvline(x=silhouette_avg1,

Clustering 85

Clustering AI AI Machine Learning 85

AWS Machine Learning Blog

APRIL 11, 2024

AWS was delighted to present to and connect with over 18,000 in-person and 267,000 virtual attendees at NVIDIA GTC, a global artificial intelligence (AI) conference that took place March 2024 in San Jose, California, returning to a hybrid, in-person experience for the first time since 2019.

AWS 131

AWS AI AI Clustering 131

ML @ CMU

NOVEMBER 7, 2024

In close collaboration with the UN and local NGOs, we co-develop an interpretable predictive tool for landmine contamination to identify hazardous clusters under geographic and budget constraints, experimentally reducing false alarms and clearance time by half. The major components of RELand are illustrated in Fig.

Clustering 153

Clustering Cross Validation Machine Learning Machine Learning 153

Hacker News

OCTOBER 15, 2024

At the Open Compute Project (OCP) Global Summit 2024, we’re showcasing our latest open AI hardware designs with the OCP community. Over the course of 2023, we rapidly scaled up our training clusters from 1K, 2K, 4K, to eventually 16K GPUs to support our AI workloads. Today, we’re training our models on two 24K-GPU clusters.

Clustering 180

Clustering AI AI Deep Learning 180

Towards AI

FEBRUARY 7, 2024

Convert your graph to a clustering-friendly format with this article. Motivation· Installing the required packages:· Assumptions· Deepwalk/Node2vec· GNNs· LINE· Apply clustering to the embeddings· Conclusion· References Using a graph can be a good way of encoding lots of information. ChatGPT, OpenAI, 30 Jan. g/g-2fkFE8rbu-dall-e.

Clustering 111

Clustering AI AI Data Science 111

PyImageSearch

SEPTEMBER 16, 2024

Home Table of Contents Credit Card Fraud Detection Using Spectral Clustering Understanding Anomaly Detection: Concepts, Types and Algorithms What Is Anomaly Detection? Spectral clustering, a technique rooted in graph theory, offers a unique way to detect anomalies by transforming data into a graph and analyzing its spectral properties.

Clustering 56

Clustering Algorithm Machine Learning Machine Learning 56

Towards AI

JANUARY 11, 2024

Last Updated on January 12, 2024 by Editorial Team Author(s): Davide Nardini Originally published on Towards AI. This often occurs in Cluster Analysis, where we identify clusters without prior information. Arguably, one of the most important concepts in machine learning is classification.

Machine Learning 130

Machine Learning Machine Learning Decision Trees Supervised Learning 130

IBM Journey to AI blog

APRIL 29, 2024

VeloxCon 2024 , the premier developer conference that is dedicated to the Velox open-source project, brought together industry leaders, engineers, and enthusiasts to explore the latest advancements and collaborative efforts shaping the future of data management.

Clustering 78

Clustering SQL Data Engineering Data Engineer 78

Hacker News

OCTOBER 28, 2024

This is a joint talk delievered by ApeCloud and China Mobile Cloud on KubeCon China 2024. This tallk introduces why and how KubeBlocks is created and how China Mobile Cloud run its cloud database without a dedicated operator.

Database 73

Database Clustering 73

DataRobot

DECEMBER 28, 2021

Yes, data created over the next three years will far exceed the amount created over the past 30 years ( Source : IDC Worldwide Global DataSphere Forecast, 2020-2024). Clustering is a technique that can be used to get a sense of the data while allowing to tell a powerful story. Introducing Multimodal Clustering. Name Clusters.

Clustering 52

Clustering Data Scientist Data Science AI 52

Dataconomy

SEPTEMBER 17, 2024

But if that wasn’t enough to make tech enthusiasts’ jaws drop, Musk recently took to his platform, X, to reveal that the real showstopper—Colossus, a 100,000 H100 training cluster—has officially come online. What exactly are AI clusters? This weekend, the @xAI team brought our Colossus 100k H100 training cluster online.

Clustering 103

Clustering AI AI Artificial Intelligence 103

Hacker News

APRIL 26, 2024

To that end, several companies are developing silicon photonics solutions, including fab providers like TSMC, who this week outlined its 3D Optical Engine roadmap as part of its 2024 North American Technology Symposium, laying out its plan to bring up to 12.8 Tbps optical connectivity to TSMC-fabbed processors.

Clustering 181

Clustering 181

Hacker News

AUGUST 5, 2024

This week at ACM SIGCOMM 2024 in Sydney, Australia, we are sharing details on the network we have built at Meta over the past few years to support our large-scale distributed AI training workload. When Meta introduced distributed GPU-based training , we decided to construct specialized data center networks tailored for these GPU clusters.

AI 175

AI AI Clustering Natural Language Processing 175

ODSC - Open Data Science

DECEMBER 21, 2023

Evaluating Clustering in Machine Learning In this article, we’ll examine two renowned clustering evaluation methods: the Silhouette score and Density-Based Clustering Validation (DBCV). 7 Data Science & AI Trends That Will Define 2024 2023 was a huge year for artificial intelligence, and 2024 will be even bigger.

Data Science 98

Data Science Clustering AI AI 98

AWS Machine Learning Blog

APRIL 1, 2024

Distributed model training requires a cluster of worker nodes that can scale. The following scaling chart shows that the p5.48xlarge instances offer 87% scaling efficiency with FSDP Llama2 fine-tuning in a 16-node cluster configuration. The example will also work with a pre-existing EKS cluster. Cluster with p4de.24xlarge

Clustering 135

Clustering AWS ML ML 135

AWS Machine Learning Blog

NOVEMBER 22, 2024

Although QLoRA helps optimize memory during fine-tuning, we will use Amazon SageMaker Training to spin up a resilient training cluster, manage orchestration, and monitor the cluster for failures. To take complete advantage of this multi-GPU cluster, we use the recent support of QLoRA and PyTorch FSDP. 24xlarge compute instance.

Clustering 94

Clustering AWS ML ML 94

Towards AI

APRIL 29, 2024

Last Updated on April 30, 2024 by Editorial Team Author(s): Harpreet Sahota Originally published on Towards AI. You’ll sign up for a Qdrant cloud account, install the necessary libraries, set up our environment variables, and instantiate a cluster — all the necessary steps to start building something. Click on the “Clusters” menu item.

Database 105

Database Clustering Python AI 105

AWS Machine Learning Blog

NOVEMBER 19, 2024

Explore the model pre-training workflow from start to finish, including setting up clusters, troubleshooting convergence issues, and running distributed training to improve model performance. In this builders’ session, learn how to pre-train an LLM using Slurm on SageMaker HyperPod.

AWS 108

AWS ML ML AI 108

Dataconomy

DECEMBER 13, 2024

billion for fiscal year 2024, driven by a 220% surge in AI revenue and a 51% increase in quarterly revenue. billion revenue, AI growth surges 220% For Q4 2024, Broadcom’s revenue reached $14.1 In fiscal 2024, semiconductor revenue was $30.1 billion in fiscal 2024, a remarkable uptick from $3.8 Broadcom Inc.

AI 91

AI AI Clustering 91

Towards AI

NOVEMBER 1, 2024

Last Updated on November 1, 2024 by Editorial Team Author(s): Get The Gist Originally published on Towards AI. Plus: Parallels Brings Apple Intelligence to Windows This member-only story is on us. Upgrade to access all of Medium.

Clustering 111

Clustering AI AI Artificial Intelligence 111

Towards AI

JANUARY 28, 2024

Last Updated on January 29, 2024 by Editorial Team Author(s): Reinhard Sellmair Originally published on Towards AI. These clusters are then one-hot-encoded and added as features. To initialize the pipeline the list of thresholds, a classifier object and the number of location clusters needs to be provided.

Clustering 115

Clustering AI AI Machine Learning 115

Towards AI

FEBRUARY 29, 2024

Last Updated on February 29, 2024 by Editorial Team Author(s): Hira Akram Originally published on Towards AI. It communicates with the Cluster Manager to allocate resources and oversee task progress. SparkContext: Facilitates communication between the Driver program and the Spark Cluster.

Apache Kafka 142

Apache Kafka SQL Clustering Data Pipeline 142

Pickl AI

MARCH 7, 2024

Summary: In 2024, mastering essential Data Science tools will be pivotal for career growth and problem-solving prowess. Top 10 Data Science tools for 2024 Are you curious about exploring Data Science tools in 2024? Platforms like Pickl.AI It provides a range of supervised and unsupervised learning algorithms.

Data Science 52

Data Science Machine Learning Machine Learning Python 52

Towards AI

APRIL 8, 2024

Last Updated on April 8, 2024 by Editorial Team Author(s): Eashan Mahajan Originally published on Towards AI. Unsupervised machine learning is generally used for clustering data. Instead, the algorithm will attempt to find functions that can be changed into simialr clusters.

Supervised Learning 105

Supervised Learning Machine Learning Machine Learning Clustering 105

Towards AI

JUNE 19, 2024

Last Updated on June 22, 2024 by Editorial Team Author(s): Stephen Chege-Tierra Insights Originally published on Towards AI. – Algorithms: K-means Clustering, ISODATA. Use Cases: Initial data exploration, finding natural clusters in data. Deciding What Algorithm to Use for Earth Observation. filterBounds(aoi).median().clip(aoi);//

Algorithm 81

Algorithm Clustering Support Vector Machines Machine Learning 81

ODSC - Open Data Science

JANUARY 3, 2024

We are kicking off 2024 in style with our ODSC East Pre-Bootcamp primer courses ! This year we have 3 new courses: Top AI Skills for 2024, Introduction to Machine Learning, and Introduction to Large Language Models and Prompt Engineering. Check out all of the sessions below.

Data Wrangling 52

Data Wrangling Machine Learning Machine Learning SQL 52

Dataconomy

DECEMBER 18, 2024

This year, tech companies collectively spent tens of billions of dollars on data centers equipped with Nvidia chips, with forecasts suggesting an estimated $229 billion in spending on servers in 2024. Microsoft alone is expected to contribute $31 billion to this total. Additionally, Amazon is developing its Trainium and Inferentia chips.

Azure 91

Azure AI AI System Architecture 91

Hacker News

NOVEMBER 12, 2023

We delivered more than what was promised—a 103 percent reduction in time-to-train for a 384-accelerator cluster.” That chip will be in volume production in 2024 and will be built using the same semiconductor manufacturing process as the Nvidia H100. On the new image generation benchmark, Gaudi 2 was also about half the H100’s speed.

AI 181

AI AI Cloud Computing Azure 181

Ocean Protocol

SEPTEMBER 9, 2024

F1 :: 2024 Strategy Analysis Poster ‘The Formula 1 Racing Challenge’ challenges participants to analyze race strategies during the 2024 season. Data The dataset includes detailed lap-by-lap data for the 2024 Formula 1 season, capturing key variables such as lap times, tire compounds, pit stop timings, stint lengths, and race positions.

EDA 45

EDA Exploratory Data Analysis Hypothesis Testing Data Science 45

Towards AI

FEBRUARY 1, 2024

Last Updated on February 1, 2024 by Editorial Team Author(s): Towards AI Editorial Team Originally published on Towards AI. Master clustering with this guide covering foundation and practical use. This week, I had a fantastic discussion with Mariam Brian, an amazing artist and CEO of Holo Art.

AI 111

AI AI Data Mining Data Mining 111

Towards AI

JUNE 12, 2024

Last Updated on June 13, 2024 by Editorial Team Author(s): Stephen Chege-Tierra Insights Originally published on Towards AI. train(training);var result = image.cluster(clusterer);// Define visualization parameters.var visParams = { min: 0, […] var roi = ee.Geometry.Rectangle([73.0, filterBounds(roi).median().clip(roi);//

Machine Learning 113

Machine Learning Machine Learning Clustering Algorithm 113

Data Science Dojo

JUNE 10, 2024

Data scientists are continuously advancing with AI tools and technologies to enhance their capabilities and drive innovation in 2024. – Example: Data scientists can use Scikit-learn for clustering customer data to identify distinct customer segments based on their purchasing behavior. H2O.ai: – H2O.ai

Data Scientist 195

Data Scientist Natural Language Processing Machine Learning Machine Learning 195

Tableau

APRIL 1, 2024

Candice Vu April 1, 2024 - 10:43pm Sanjeev Verma Product Management Senior Manager In today's data and AI-driven world, it’s important to have the right tools to navigate and analyze vast data sources. Tableau will monitor the health of the cluster and software client to take appropriate action if they are in an unhealthy state.

Tableau 70

Tableau Clustering Cloud Data AI 70