# US 2026/0087023 A1 · Re-ranking and outlier detection in an augmented semantic search system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0087023 A1
- Document ID: US20260087023A1
- Status: Published application
- Kind: A1
- Application number: 18/936,950
- Filed: 2024-11-04
- Published: 2026-03-26
- Earliest claimed: 2024-09-26
- Assignees: Tubi, Inc.
- Applicants: Tubi, Inc.
- Inventors: John Trenkle; Jaya Kawale; Blake Scott Bassett; Bethany Marie Baker; Claire Elise Dorman; Fenglin Yuan
- Web page: https://patentgrep.com/patent/US20260087023A1

## CPC

- G06F 16/435: Filtering based on additional data, e.g. user or group profiles
- G06F 16/2237: {Vectors, bitmaps or matrices}
- G06F 16/2365: {Ensuring data consistency and integrity}
- G06F 16/24565: {Triggers; Constraints}
- G06F 16/24578: {using ranking}
- G06F 16/41: Indexing; Data structures therefor; Storage structures
- G06F 16/44: Browsing; Visualisation therefor
- G06F 16/45: Clustering; Classification
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A system and method for augmented semantic search, including: a vector store including a set of embeddings, each representing a structured data representation of a media perspective of a media item; a query execution service including functionality to receive a search request including a query string from a client application; a query classification service including functionality to execute a first machine learning model to generate a classification object in a structured classification format; a filter extraction service including functionality to execute a second machine learning model to generate a filter object including a set of filters in a structured filter format; a recaller service including functionality to: execute an encoder model on the input query and execute a vector similarity operation on the query embedding to generate a result set; and a re-ranking service including functionality to: execute a large language model to re-rank the result set.

## Claims

### Claim 1

A system for semantic search, comprising:
a computer processor;
a vector store comprising a set of embeddings positioned within a unified vector space. wherein multiple of the set of embeddings correspond to a single media item, each representing a different structured data representation of a media perspective of the media item;
a query execution service comprising functionality to:
receive a search request comprising a query string from a client application;
a query classification service comprising functionality to:
execute a first machine learning model to generate a classification object representing classification of the query string in a structured classification format;
a filter extraction service comprising functionality to:
execute a second machine learning model to generate a filter object comprising a set of filters inferred for the query string in a structured filter format;
a recaller service comprising functionality to:
execute an encoder model on the input query, incorporating the classification object and the filter object, to generate a query embedding representing the input query in the unified vector space;
use the filter object to identify a constrained set of candidate embeddings of the vector store; and
execute a vector similarity operation within the unified vector space on the query embedding and the constrained set of candidate embeddings to generate a match set of embeddings; and
a re-ranking service comprising functionality to:
generate a re-ranking prompt comprising the query embedding and the match set of embeddings;
execute a large language model using the re-ranking prompt to generate a re-ranked match set of embeddings; and
provide the re-ranked match set of embeddings in response to the search request.

### Claim 2 (depends on 1)

The system of claim 1, wherein the re-ranking service is further configured to perform outlier detection by:
generating an outlier detection prompt comprising the query embedding, the match set of embeddings, and contextual information derived from the classification object and filter object;
executing the large language model using the outlier detection prompt to generate an outlier score for each embedding in the match set;
applying a dynamic outlier threshold based on the classification object and the distribution of outlier scores;
identifying embeddings with outlier scores exceeding the dynamic outlier threshold;
analyzing semantic relationships between identified outlier embeddings and non-outlier embeddings;
selectively excluding outlier embeddings based on both their outlier scores and their semantic distance from non-outlier embeddings; and
adjusting the re-ranked match set of embeddings to exclude the selectively excluded outlier embeddings.

### Claim 3 (depends on 1)

The system of claim 1, wherein the re-ranking service is further configured to:
incorporate additional inputs into the re-ranking prompt, the additional inputs comprising at least one selected from a group consisting of: user profile data, historical search behavior, trending topics, and contextual information.

### Claim 4 (depends on 1)

The system of claim 1, wherein the re-ranking service is further configured to:
incorporate the classification object and filter object into the re-ranking prompt;
generate a relevance score for each embedding based on its alignment with both the classification object and the filter object;
apply a weighted ranking algorithm balancing classification alignment and filter adherence;
prioritize embeddings matching both classification intent and filter criteria; and
ensure result diversity by considering secondary classifications when primary classification intent is met.

### Claim 5 (depends on 1)

The system of claim 1, further comprising:
an autodata generation service comprising functionality to generate the structured data representations for each media item by:
extracting caption data from the media item;
generating an autodata prompt by analyzing the caption data to identify key themes, entities, and contexts;
selecting relevant prompt templates based on the media item's type and content;
executing a third large language model using the autodata prompt to generate the structured data representation; and
validating the generated representation for accuracy and completeness.

### Claim 6 (depends on 1)

The system of claim 1, wherein:
the query classification system executes a second large language model to generate the classification object by mapping the query to predefined classification categories;
the filter extraction system executes a third large language model to generate the filter object by identifying specific entities within the query string using boolean and range-based parameters;
the classification object informs the third large language model to refine filter granularity and resolve ambiguities.

### Claim 7 (depends on 1)

The system of claim 1, further comprising a multi-classification analyzer service configured to:
analyze the query string to identify multiple distinct classifications;
for each identified classification:
generate a classification-specific query embedding using the encoder model,
cause the recaller service to execute a separate vector similarity operation on the classification-specific query embedding to generate a classification-specific match set of embeddings, and
apply the re-ranking service to the classification-specific match set of embeddings to generate a classification-specific re-ranked match set;
merge the classification-specific re-ranked match sets into a compound result set by:
assigning weights to each classification-specific re-ranked match set based on relevance scores derived from the query classification service,
normalizing ranking scores across all classification-specific re-ranked match sets,
interleaving results from each classification-specific re-ranked match set based on the normalized ranking scores and classification weights, and
applying a diversity algorithm to ensure representation from each identified classification in the compound result set; and
provide the compound result set in response to the search request.

### Claim 8 (depends on 1)

The system of claim 1, wherein the re-ranking service is further configured to:
generate, for each embedding in the match set of embeddings, a confidence score indicating a likelihood of relevance to the query string;
apply a dynamic threshold to the confidence scores, wherein the dynamic threshold is adjusted based on the classification object and filter object; and
include in the re-ranked match set of embeddings only those embeddings exceeding the dynamic threshold.

### Claim 9 (depends on 1)

The system of claim 1, wherein the re-ranking service is further configured to:
identify semantic relationships between embeddings in the match set;
cluster semantically related embeddings; and
adjust the ranking of embeddings within each cluster to ensure diversity in the re-ranked match set of embeddings.

### Claim 10 (depends on 1)

The system of claim 1, further comprising an adaptive learning module comprising functionality to:
analyze user interactions with the re-ranked match set of embeddings;
generate labeled training examples from search interactions;
fine-tune the large language model using reinforcement learning techniques;
adjust the large language model's ranking behavior based on evolving user preferences; and
periodically evaluate the fine-tuned large language model to ensure improved performance across various domains and user segments.

### Claim 11

A method for semantic search, comprising:
receiving a search request comprising a query string from a client application;
executing a first machine learning model to generate a classification object representing classification of the query string in a structured classification format;
executing a second machine learning model to generate a filter object comprising a set of filters inferred for the query string in a structured filter format;
executing an encoder model on the input query, incorporating the classification object and the filter object, to generate a query embedding representing the input query in a unified vector space;
using the filter object to identify a constrained set of candidate embeddings of a vector store comprising a set of embeddings, wherein the constrained set of embeddings is a subset of the set of embeddings positioned within the unified vector space, wherein multiple of the set of embeddings in the unified vector space correspond to a single media item, each representing a different structured data representation of a media perspective of the media item;
executing a vector similarity operation within the unified vector space on the query embedding and the constrained set of candidate embeddings to generate a match set of embeddings;
generating a re-ranking prompt comprising the query embedding and the match set of embeddings;
executing, by a computer processor, a large language model using the re-ranking prompt to generate a re-ranked match set of embeddings; and
providing the re-ranked match set of embeddings in response to the search request.

### Claim 12 (depends on 11)

The method of claim 11, further comprising:
generating an outlier detection prompt comprising the query embedding, the match set of embeddings, and contextual information derived from the classification object and filter object;
executing the large language model using the outlier detection prompt to generate an outlier score for each embedding in the match set;
applying a dynamic outlier threshold based on the classification object and the distribution of outlier scores;
identifying embeddings with outlier scores exceeding the dynamic outlier threshold;
analyzing semantic relationships between identified outlier embeddings and non-outlier embeddings;
selectively excluding outlier embeddings based on both their outlier scores and their semantic distance from non-outlier embeddings; and
adjusting the re-ranked match set of embeddings to exclude the selectively excluded outlier embeddings.

### Claim 13 (depends on 11)

The method of claim 11, further comprising:
incorporating additional inputs into the re-ranking prompt, the additional inputs comprising at least one selected from a group consisting of: user profile data, historical search behavior, trending topics, and contextual information.

### Claim 14 (depends on 11)

The method of claim 11, further comprising:
incorporating the classification object and filter object into the re-ranking prompt;
generating a relevance score for each embedding based on its alignment with both the classification object and the filter object;
applying a weighted ranking algorithm balancing classification alignment and filter adherence;
prioritizing embeddings matching both classification intent and filter criteria; and
ensuring result diversity by considering secondary classifications when primary classification intent is met.

### Claim 15 (depends on 11)

The method of claim 11, further comprising:
generating the structured data representations for each media item by:
extracting caption data from the media item;
generating an autodata prompt by analyzing the caption data to identify key themes, entities, and contexts;
selecting relevant prompt templates based on the media item's type and content;
executing a third large language model using the autodata prompt to generate the structured data representation; and
validating the generated representation for accuracy and completeness.

### Claim 16 (depends on 11)

The method of claim 11, further comprising:
executing a second large language model to generate the classification object by mapping the query to predefined classification categories; and
executing a third large language model to generate the filter object by identifying specific entities within the query string using boolean and range-based parameters, wherein the classification object informs the third large language model to refine filter granularity and resolve ambiguities.

### Claim 17 (depends on 11)

The method of claim 11, further comprising:
analyzing the query string to identify multiple distinct classifications;
for each identified classification:
generating a classification-specific query embedding using the encoder model,
causing the recaller service to execute a separate vector similarity operation on the classification-specific query embedding to generate a classification-specific match set of embeddings, and
applying the re-ranking service to the classification-specific match set of embeddings to generate a classification-specific re-ranked match set;
merging the classification-specific re-ranked match sets into a compound result set by:
assigning weights to each classification-specific re-ranked match set based on relevance scores derived from the query classification service,
normalizing ranking scores across all classification-specific re-ranked match sets,
interleaving results from each classification-specific re-ranked match set based on the normalized ranking scores and classification weights, and
applying a diversity algorithm to ensure representation from each identified classification in the compound result set; and
providing the compound result set in response to the search request.

### Claim 18 (depends on 11)

The method of claim 11, further comprising:
generating, for each embedding in the match set of embeddings, a confidence score indicating a likelihood of relevance to the query string;
applying a dynamic threshold to the confidence scores, wherein the dynamic threshold is adjusted based on the classification object and filter object; and
including in the re-ranked match set of embeddings only those embeddings exceeding the dynamic threshold.

### Claim 19 (depends on 11)

The method of claim 11, further comprising:
identifying semantic relationships between embeddings in the match set;
clustering semantically related embeddings; and
adjusting the ranking of embeddings within each cluster to ensure diversity in the re-ranked match set of embeddings.

### Claim 20

A non-transitory computer-readable storage medium comprising a plurality of instructions for semantic search, the plurality of instructions configured to execute on at least one computer processor to enable the at least one computer processor to:
receive a search request comprising a query string from a client application;
execute a first machine learning model to generate a classification object representing classification of the query string in a structured classification format;
execute a second machine learning model to generate a filter object comprising a set of filters inferred for the query string in a structured filter format;
execute an encoder model on the input query, incorporating the classification object and the filter object, to generate a query embedding representing the input query in a unified vector space;
use the filter object to identify a constrained set of candidate embeddings of a vector store comprising a set of embeddings, wherein the constrained set of embeddings is a subset of the set of embeddings positioned within the unified vector space, wherein multiple of the set of embeddings in the unified vector space correspond to a single media item, each representing a different structured data representation of a media perspective of the media item;
execute a vector similarity operation within the unified vector space on the query embedding and the constrained set of candidate embeddings to generate a match set of embeddings;
generate a re-ranking prompt comprising the query embedding and the match set of embeddings;
execute a large language model using the re-ranking prompt to generate a re-ranked match set of embeddings; and
provide the re-ranked match set of embeddings in response to the search request.
