AWS Machine Learning Blog

MARCH 11, 2025

Vector database FloTorch selected Amazon OpenSearch Service as a vector database for its high-performance metrics. Retrieval (and reranking) strategy FloTorch used a retrieval strategy with a k-nearest neighbor (k-NN) of five for retrieved chunks. Each provisioned node was r7g.4xlarge,

K-nearest Neighbors 94

K-nearest Neighbors AWS Database AI 94

Towards AI

FEBRUARY 19, 2025

Vector Databases 101: A Beginners Guide to Vector Search and Indexing Photo by Google DeepMind on Unsplash Introduction Alright, folks! The secret sauce behind all of this is vector search and vector databases, helping power similarity-based recommendations and retrieval! Traditional databases? They tap out.

Database 70

Database K-nearest Neighbors Machine Learning Machine Learning 70

AWS Machine Learning Blog

FEBRUARY 13, 2025

Caching is performed on Amazon CloudFront for certain topics to ease the database load. Amazon Aurora PostgreSQL-Compatible Edition and pgvector Amazon Aurora PostgreSQL-Compatible is used as the database, both for the functionality of the application itself and as a vector store using pgvector. Its hosted on AWS Lambda.

AWS 94

AWS K-nearest Neighbors Clustering Algorithm 94

AWS Machine Learning Blog

FEBRUARY 5, 2025

These databases typically use k-nearest (k-NN) indexes built with advanced algorithms such as Hierarchical Navigable Small Worlds (HNSW) and Inverted File (IVF) systems. OpenSearch Service then uses the vectors to find the k-nearest neighbors (KNN) to the vectorized search term and image to retrieve the relevant listings.

K-nearest Neighbors 76

K-nearest Neighbors Machine Learning Machine Learning Database 76

Flipboard

FEBRUARY 7, 2025

This post shows you how to set up RAG using DeepSeek-R1 on Amazon SageMaker with an OpenSearch Service vector database as the knowledge base. When combined with Amazon OpenSearch Service , it enables robust Retrieval Augmented Generation (RAG) applications.

Database 87

Database AWS Python ML 87

AWS Machine Learning Blog

SEPTEMBER 8, 2023

You then use Exact k-NN with scoring script so that you can search by two fields: celebrity names and the vector that captured the semantic information of the article. You also generate an embedding of this newly written article, so that you can search OpenSearch Service for the nearest images to the article in this vector space.

K-nearest Neighbors 116

K-nearest Neighbors AWS ML ML 116

AWS Machine Learning Blog

APRIL 19, 2024

We stored the embeddings in a vector database and then used the Large Language-and-Vision Assistant (LLaVA 1.5-7b) 7b) model to generate text responses to user questions based on the most similar slide retrieved from the vector database. OpenSearch Serverless is an on-demand serverless configuration for Amazon OpenSearch Service.

AWS 121

AWS ML ML Database 121

AWS Machine Learning Blog

APRIL 3, 2024

We detail the steps to use an Amazon Titan Multimodal Embeddings model to encode images and text into embeddings, ingest embeddings into an OpenSearch Service index, and query the index using the OpenSearch Service k-nearest neighbors (k-NN) functionality. These steps are completed prior to the user interaction steps.

K-nearest Neighbors 119

K-nearest Neighbors AWS Machine Learning Machine Learning 119

IBM Journey to AI blog

DECEMBER 20, 2023

Classification algorithms include logistic regression, k-nearest neighbors and support vector machines (SVMs), among others. Association algorithms allow data scientists to identify associations between data objects inside large databases, facilitating data visualization and dimensionality reduction.

Machine Learning 92

Machine Learning Machine Learning Supervised Learning Clustering 92

AWS Machine Learning Blog

JANUARY 30, 2024

We use OpenSearch Serverless as a vector database for storing embeddings generated by the Titan Multimodal Embeddings model. In the user interaction phase, a question from the user is converted into embeddings and a similarity search is run on the vector database to find a slide that could potentially contain answers to user question.

AWS 119

AWS ML ML K-nearest Neighbors 119

PyImageSearch

JANUARY 27, 2025

Home Table of Contents Approximate Nearest Neighbor with Locality Sensitive Hashing (LSH) What Is Locality Sensitive Hashing (LSH)? Refinement: The candidate set is then refined by computing the actual distances between the query point and the candidates to find the approximate nearest neighbors.

K-nearest Neighbors 82

K-nearest Neighbors Algorithm Data Preparation Database 82

Pickl AI

JULY 17, 2023

It aims to partition a given dataset into K clusters, where each data point belongs to the cluster with the nearest mean. K-NN (k nearest neighbors): K-Nearest Neighbors (K-NN) is a simple yet powerful algorithm used for both classification and regression tasks in Machine Learning.

Machine Learning 52

Machine Learning Machine Learning K-nearest Neighbors Clustering 52

AWS Machine Learning Blog

DECEMBER 6, 2023

Another driver behind RAG’s popularity is its ease of implementation and the existence of mature vector search solutions, such as those offered by Amazon Kendra (see Amazon Kendra launches Retrieval API ) and Amazon OpenSearch Service (see k-Nearest Neighbor (k-NN) search in Amazon OpenSearch Service ), among others.

SQL 121

SQL AWS Analytics Analytics 121

AWS Machine Learning Blog

AUGUST 26, 2024

You can integrate existing data from AWS data lakes, Amazon Simple Storage Service (Amazon S3) buckets, or Amazon Relational Database Service (Amazon RDS) instances with services such as Amazon Bedrock and Amazon Q. The Asynchronous Request Handler function stores results in a DynamoDB database along with the generated requestId.

AWS 119

AWS AI AI K-nearest Neighbors 119

AWS Machine Learning Blog

FEBRUARY 25, 2025

Instead of treating each input as entirely unique, we can use a distance-based approach like k-nearest neighbors (k-NN) to assign a class based on the most similar examples surrounding the input. To make this work, we need to transform the textual interactions into a format that allows algebraic operations.

Algorithm 70

Algorithm Machine Learning Machine Learning K-nearest Neighbors 70

AWS Machine Learning Blog

JUNE 3, 2024

You store the embeddings of the video frame as a k-nearest neighbors (k-NN) vector in your OpenSearch Service index with the reference to the video clip and the frame in the S3 bucket itself (Step 3). You split the video files into frames and save them in a S3 bucket (Step 1).

AWS 103

AWS K-nearest Neighbors ML ML 103

AWS Machine Learning Blog

JANUARY 6, 2023

In this post, we present a solution to handle OOC situations through knowledge graph-based embedding search using the k-nearest neighbor (kNN) search capabilities of OpenSearch Service. Solution overview. The key AWS services used to implement this solution are OpenSearch Service, SageMaker, Lambda, and Amazon S3.

AWS 96

AWS ML ML Machine Learning 96

Mlearning.ai

NOVEMBER 29, 2023

K-Nearest Neighbor Regression Neural Network (KNN) The k-nearest neighbor (k-NN) algorithm is one of the most popular non-parametric approaches used for classification, and it has been extended to regression. Decision Trees ML-based decision trees are used to classify items (products) in the database.

Machine Learning 52

Machine Learning Machine Learning ML ML 52

AWS Machine Learning Blog

SEPTEMBER 25, 2024

#LuxuryBrand #TimelessElegance #ExclusiveCollection Retrieve and analyze the top three relevant posts The next step involves using the generated image and text to search for the top three similar historical posts from a vector database. The following code snippet shows the implementation of this step.

AWS 85

AWS K-nearest Neighbors ML ML 85

Pickl AI

SEPTEMBER 12, 2024

Key Components of Data Science Data Science consists of several key components that work together to extract meaningful insights from data: Data Collection: This involves gathering relevant data from various sources, such as databases, APIs, and web scraping. Data Cleaning: Raw data often contains errors, inconsistencies, and missing values.

Data Analyst 52

Data Analyst Data Science Machine Learning Machine Learning 52

Snorkel AI

APRIL 28, 2023

On one hand, there’s a data management community trying to understand data transformation and computing some functions over exponentially many databases for decades. You can approximate your machine learning training components into some simpler classifiers—for example, a k-nearest neighbors classifier.

ML 52

ML ML Machine Learning Machine Learning 52

Snorkel AI

APRIL 28, 2023

On one hand, there’s a data management community trying to understand data transformation and computing some functions over exponentially many databases for decades. You can approximate your machine learning training components into some simpler classifiers—for example, a k-nearest neighbors classifier.

ML 52

ML ML Machine Learning Machine Learning 52

DagsHub

JUNE 24, 2024

Source: [link] The previous system works this way: there is a bank of face images, their corresponding embeddings are stored in a vector database and the labels are also available.

Clustering 40

Clustering Machine Learning Machine Learning K-nearest Neighbors 40

Pickl AI

NOVEMBER 18, 2024

Structured data refers to neatly organised data that fits into tables, such as spreadsheets or databases, where each column represents a feature and each row represents an instance. This data can come from databases, APIs, or public datasets. K-Nearest Neighbors), while others can handle large datasets efficiently (e.g.,

Machine Learning 52

Machine Learning Machine Learning Decision Trees Algorithm 52

The MLOps Blog

DECEMBER 19, 2022

Adding vectors to the index (xb are database vectors that are to be indexed). index.add(xb) # xq are query vectors, for which we need to search in xb to find the k nearest neighbors. # The search returns D, the pairwise distances, and I, the indices of the nearest neighbors. . # Creating the index.

ML 52

ML ML Algorithm Deep Learning 52

Mlearning.ai

MAY 23, 2023

The K-Nearest Neighbor Algorithm is a good example of an algorithm with low bias and high variance. This trade-off can easily be reversed by increasing the k value which in turn results in increasing the number of neighbours. Let us see some examples. So, prepare well for that as well.

Data Science 52

Data Science Decision Trees Machine Learning Machine Learning 52

Flipboard

JANUARY 24, 2025

He leads the product initiatives for AI and machine learning (ML) on OpenSearch including OpenSearchs vector database capabilities. Dylan has decades of experience working directly with customers and creating products and solutions in the database, analytics and AI/ML domain.

K-nearest Neighbors 145

K-nearest Neighbors ML ML Algorithm 145

AWS Machine Learning Blog

DECEMBER 10, 2024

We stored the embeddings in a vector database and then used the Large Language-and-Vision Assistant (LLaVA 1.5-7b) 7b) model to generate text responses to user questions based on the most similar slide retrieved from the vector database. She is passionate about sharing knowledge and fostering interest in emerging talent.

AWS 96

AWS K-nearest Neighbors Database ML 96

Dataconomy

DECEMBER 12, 2024

Databases to be migrated can have a wide range of data representations and contents. For the sake of argument, let’s ignore the fact that the use of such data types in databases is justified only in a few specific cases, as this problem often arises when migrating complex systems. in XML, CLOB, BLOB etc.

K-nearest Neighbors 91

K-nearest Neighbors Machine Learning Machine Learning Support Vector Machines 91

AWS Machine Learning Blog

OCTOBER 24, 2024

The second post outlines how to work with multiple data formats such as structured data (tables, databases) and images. In traditional RAG use cases, the chatbot relies on a database of text documents (.doc,pdf, Practically, this can be achieved in OpenSearch by combining a k-nearest neighbors (k-NN) query with keyword matching.

AWS 82

AWS K-nearest Neighbors Database AI 82