# US 2026/0087065 A1 · Classification and filtering in an augmented semantic search system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0087065 A1
- Document ID: US20260087065A1
- Status: Published application
- Kind: A1
- Application number: 18/936,930
- Filed: 2024-11-04
- Published: 2026-03-26
- Earliest claimed: 2024-09-26
- Assignees: Tubi, Inc.
- Applicants: Tubi, Inc.
- Inventors: John Trenkle; Jaya Kawale; Blake Scott Bassett; Bethany Marie Baker; Claire Elise Dorman; Fenglin Yuan
- Web page: https://patentgrep.com/patent/US20260087065A1

## CPC

- G06F 16/435: Filtering based on additional data, e.g. user or group profiles
- G06F 16/2237: {Vectors, bitmaps or matrices}
- G06F 16/2365: {Ensuring data consistency and integrity}
- G06F 16/24565: {Triggers; Constraints}
- G06F 16/24578: {using ranking}
- G06F 16/41: Indexing; Data structures therefor; Storage structures
- G06F 16/44: Browsing; Visualisation therefor
- G06F 16/45: Clustering; Classification
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A system and method for augmented semantic search, including: a query execution service including functionality to receive a search request including a query string; a query classification service including functionality to execute a first large language model using a first prompt to generate a classification object representing classification of the query string in a structured classification format; a filter extraction service including functionality to execute a second large language model using a second prompt to generate a filter object including a set of filters inferred for the query string in the structured filter format; and a recaller service including functionality to: use the filter object to constrain search space of a vector store and execute a vector similarity operation on a query vector to generate a match set of embeddings; and provide a result set based on the match set of embeddings.

## Claims

### Claim 1

A system for augmented semantic search, comprising:
a computer processor;
a query execution service comprising functionality to:
receive a search request comprising a query string from a client application;
a query classification service comprising functionality to:
generate a first prompt comprising the search request, a set of categories, descriptions of the set of categories, and definition of a structured classification format; and
execute a first large language model using the first prompt to generate a classification object representing classification of the query string in the structured classification format;
a filter extraction service executing on the computer processor and comprising functionality to:
generate a second prompt comprising the search request, a set of filter criteria, and definition of a structured filter format; and
execute a second large language model using the second prompt to generate a filter object comprising a set of filters inferred for the query string in the structured filter format; and
a recaller service comprising functionality to:
generate a query vector for the search request using the query string, the classification object, and the filter object;
use the filter object to identify a constrained set of candidate embeddings of a vector store;
execute a vector similarity operation on the query vector and the constrained set of candidate embeddings to generate a match set of embeddings; and
provide, in response to the search request, a result set comprising identifiers of a matching set of media items referenced by the match set of embeddings.

### Claim 2 (depends on 1)

The system of claim 1, further comprising:
an indexer service comprising functionality to:
obtain a set of structured data representations of media perspectives for a set of media items, wherein the set of structured data representations is generated based on caption data of the set of media items; and
execute an encoder model on the set of structured data representations to generate the set of embeddings stored in the vector store.

### Claim 3 (depends on 1)

The system of claim 1, wherein the query classification service is further configured to:
analyze the query string to determine that it is associated with two or more disjoint classifications;
asynchronously analyze the query string for each classification using separate instances of the first large language model; and
merge the results using a multi-classification analyzer service to provide a comprehensive classification object.

### Claim 4 (depends on 3)

The system of claim 3, wherein the multi-classification analyzer service is further configured to:
receive multiple classification objects comprising the classification object from separate instances of the first large language model;
analyze semantic relationships between the multiple classification objects;
generate a unified classification hierarchy that incorporates all identified classifications;
assign confidence scores to each classification within the unified hierarchy; and
produce a comprehensive classification object that comprises:
the unified classification hierarchy,
confidence scores for each classification,
potential conflicts or overlaps between classifications, and
a set of suggested query refinements based on the multiple classifications.

### Claim 5 (depends on 1)

The system of claim 1, wherein each of the set of media items is associated with multiple different structured data representations of different media perspectives among the set of structured data representations, the different media perspectives comprising at least two selected from a group consisting of: hierarchical summaries, character and cast analyses, setting and time classifications, thematic and tonal analyses, plot and action detections, and micro-genre classifications.

### Claim 6 (depends on 1)

The system of claim 1, wherein the classification object represents a classification of the entire query string, and the filter object comprises a set of filters inferred for specific entities or attributes within the query string.

### Claim 7 (depends on 1)

The system of claim 1, further comprising an intelligent re-ranking system configured to:
generate a re-ranking prompt comprising the query string, the classification object, the filter object, and the match set of embeddings;
execute a third large language model using the re-ranking prompt to:
analyze the relevance of each embedding in the match set to the query intent derived from the classification object,
evaluate the adherence of each embedding to the constraints specified in the filter object, and
generate a contextual importance score for each embedding based on the classification and filter analysis;
re-rank the match set of embeddings based on the contextual importance scores; and
provide the re-ranked match set of embeddings in response to the search request.

### Claim 8 (depends on 1)

The system of claim 1, wherein the filter extraction service is further configured to:
identify named entities in the query string;
map the named entities to predefined filter criteria; and
include entity-specific filter parameters in the filter object.

### Claim 9 (depends on 1)

The system of claim 1, further comprising a data validation and quality control module configured to:
analyze the classification object and filter object for consistency with historical search patterns; and
adjust the classification object or filter object if inconsistencies are detected, thereby improving the accuracy of query vector generation.

### Claim 10

A method for augmented semantic search, comprising:
receiving a search request comprising a query string from a client application;
generating a first prompt comprising the search request, a set of categories, descriptions of the set of categories, and definition of a structured classification format;
executing, by a computer processor, a first large language model using the first prompt to generate a classification object representing classification of the query string in the structured classification format;
generating a second prompt comprising the search request, a set of filter criteria, and definition of a structured filter format;
executing a second large language model using the second prompt to generate a filter object comprising a set of filters inferred for the query string in the structured filter format;
generating a query vector for the search request using the query string, the classification object, and the filter object;
using the filter object to identify a constrained set of candidate embeddings of a vector store;
executing a vector similarity operation on the query vector and the constrained set of candidate embeddings to generate a match set of embeddings; and
providing, in response to the search request, a result set comprising identifiers of a matching set of media items referenced by the match set of embeddings.

### Claim 11 (depends on 10)

The method of claim 10, further comprising:
obtaining a set of structured data representations of media perspectives for a set of media items, wherein the set of structured data representations is generated based on caption data of the set of media items; and
executing an encoder model on the set of structured data representations to generate the set of embeddings stored in the vector store.

### Claim 12 (depends on 10)

The method of claim 10, further comprising:
analyzing the query string to determine that it is associated with two or more disjoint classifications;
asynchronously analyzing the query string for each classification using separate instances of the first large language model; and
merging the results using a multi-classification analyzer service to provide a comprehensive classification object.

### Claim 13 (depends on 12)

The method of claim 12, further comprising:
receiving multiple classification objects comprising the classification object from separate instances of the first large language model;
analyzing semantic relationships between the multiple classification objects;
generating a unified classification hierarchy that incorporates all identified classifications;
assigning confidence scores to each classification within the unified hierarchy; and
producing a comprehensive classification object that comprises:
the unified classification hierarchy,
confidence scores for each classification,
potential conflicts or overlaps between classifications, and
a set of suggested query refinements based on the multiple classifications.

### Claim 14 (depends on 10)

The method of claim 10, wherein each of the set of media items is associated with multiple different structured data representations of different media perspectives among the set of structured data representations, the different media perspectives comprising at least two selected from a group consisting of: hierarchical summaries, character and cast analyses, setting and time classifications, thematic and tonal analyses, plot and action detections, and micro-genre classifications.

### Claim 15 (depends on 10)

The method of claim 10, wherein the classification object represents a classification of the entire query string, and the filter object comprises a set of filters inferred for specific entities or attributes within the query string.

### Claim 16 (depends on 10)

The method of claim 10, further comprising:
generating a re-ranking prompt comprising the query string, the classification object, the filter object, and the match set of embeddings;
executing a third large language model using the re-ranking prompt to:
analyze the relevance of each embedding in the match set to the query intent derived from the classification object,
evaluate the adherence of each embedding to the constraints specified in the filter object, and
generate a contextual importance score for each embedding based on the classification and filter analysis;
re-ranking the match set of embeddings based on the contextual importance scores; and
providing the re-ranked match set of embeddings in response to the search request.

### Claim 17 (depends on 10)

The method of claim 10, further comprising:
identifying named entities in the query string;
mapping the named entities to predefined filter criteria; and
including entity-specific filter parameters in the filter object.

### Claim 18 (depends on 10)

The method of claim 10, further comprising:
analyzing the classification object and filter object for consistency with historical search patterns; and
adjusting the classification object or filter object if inconsistencies are detected, thereby improving the accuracy of query vector generation.

### Claim 19

A non-transitory computer-readable storage medium comprising a plurality of instructions for augmented semantic search, the plurality of instructions configured to execute on at least one computer processor to enable the at least one computer processor to:
receive a search request comprising a query string from a client application;
generate a first prompt comprising the search request, a set of categories, descriptions of the set of categories, and definition of a structured classification format;
execute a first large language model using the first prompt to generate a classification object representing classification of the query string in the structured classification format;
generate a second prompt comprising the search request, a set of filter criteria, and definition of a structured filter format;
execute a second large language model using the second prompt to generate a filter object comprising a set of filters inferred for the query string in the structured filter format;
generate a query vector for the search request using the query string, the classification object, and the filter object;
use the filter object to identify a constrained set of candidate embeddings of a vector store;
execute a vector similarity operation on the query vector and the constrained set of candidate embeddings to generate a match set of embeddings; and
provide, in response to the search request, a result set comprising identifiers of a matching set of media items referenced by the match set of embeddings.

### Claim 20 (depends on 19)

The non-transitory computer-readable storage medium of claim 19, the plurality of instructions further configured to enable the at least one computer processor to:
analyze the query string to determine that it is associated with two or more disjoint classifications;
asynchronously analyze the query string for each classification using separate instances of the first large language model; and
merge the results using a multi-classification analyzer service to provide a comprehensive classification object.
