# US 2026/0087818 A1 · Image recognition based on signature analysis

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0087818 A1
- Document ID: US20260087818A1
- Status: Published application
- Kind: A1
- Application number: 19/334,008
- Filed: 2025-09-19
- Published: 2026-03-26
- Earliest claimed: 2024-09-22
- Assignees: Tyco Fire & Security GmbH
- Applicants: Tyco Fire & Security GmbH
- Inventors: Amit Rozner; Yohai Falik; Tamir Manor; Venkata Pavan Muppala; Rajkiran Kumar Gottumukkal
- Web page: https://patentgrep.com/patent/US20260087818A1

## CPC

- G06F 16/587: using geographical or spatial information, e.g. location
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06F 16/51: Indexing; Data structures therefor; Storage structures
- G06F 16/532: Query formulation, e.g. graphical querying
- G06F 16/583: using metadata automatically derived from the content
- G06F 40/00: Handling natural language data (speech analysis or synthesis, speech recognition G10L)
- G06F 40/20: Natural language analysis (semantic analysis of natural language G06F40/30)
- G06F 40/30: Semantic analysis
- G06V 10/7747: {Organisation of the process, e.g. bagging or boosting}
- G06V 10/776: Validation; Performance evaluation
- G06F 40/40: Processing or translation of natural language (natural language analysis G06F40/20; semantic analysis G06F40/30)
- G06V 10/82: using neural networks
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A system can include one or more memory devices that can store instructions thereon. The instructions can, when executed by one or more processors, cause the one or more processors to receive image data, extract a plurality of image frames from the image data, generate a plurality of image signatures that describe features within the plurality of image frames, store the plurality of image signatures in a database, receive a natural language query, generate a textual signature that describes the natural language query, and perform a search of the database for one or more matches between the textual signature and the plurality of image signatures.

## Claims

### Claim 1

A system comprising one or more memory devices storing instructions thereon that, when executed by one or more processors, cause the one or more processors to:
receive, from one or more cameras of a building, image data;
extract a plurality of image frames from the image data;
generate, using a first machine learning model, a plurality of image signatures that describe features within the plurality of image frames;
store, responsive to generation of the plurality of image signatures, the plurality of image signatures in a database;
receive, from a user device, a natural language query;
generate, using a second machine learning model, a textual signature that describes the natural language query; and
perform, responsive to generation of the textual signature, a search of the database for one or more matches between the textual signature and the plurality of image signatures.

### Claim 2 (depends on 1)

The system of claim 1, wherein the instructions further cause the one or more processors to:
detect the one or more matches between the textual signature and at least one image signature of the plurality of image signatures;
identify at least one image frame of the plurality of image frames that is described by the at least one image signature; and
output, for display by a display device, the at least one image frame.

### Claim 3 (depends on 1)

The system of claim 1, wherein the first machine learning model includes an image encoder, wherein the second machine learning model includes a text encoder, and wherein the instructions further cause the one or more processors to train the image encoder and the text encoder by:
compiling one or more sets of training data that include (i) one or more textual inputs and (ii) one or more image inputs, wherein the one or more textual inputs and the one or more image inputs both describe one or more training image frames;
providing the one or more textual inputs to the text encoder to cause the text encoder to generate one or more textual signatures;
providing the one or more image inputs to the image encoder to cause the image encoder to generate one or more image signatures;
detecting, based on a comparison between the one or more textual signatures and the one or more image signatures, that an amount of variance between the image encoder and the text encoder adheres to one or more aspects of contrastive loss-based training; and
deploying the first machine learning model and the second machine learning model.

### Claim 4 (depends on 1)

The system of claim 1, wherein the instructions further cause the one or more processors to:
while the first machine learning model is being trained:
provide, to the second machine learning model, one or more textual inputs that provide a textual description of at least one image frame, wherein the one or more textual inputs cause the second machine learning model to output one or more textual signatures that describe the at least one image frame;
provide, to the first machine learning model, the at least one image frame to cause the first machine learning model to output one or more image signatures that describe the at least one image frame; and
determine a performance of the first machine learning model based on a difference between the one or more textual signatures and the one or more image signatures.

### Claim 5 (depends on 1)

The system of claim 1, wherein the instructions further cause the one or more processors to:
detect that the natural language query includes an indication of one or more points in time or a particular zone within the building;
identify, based on the search of the database, the one or more matches, wherein the one or more matches are between the textual signature and one or more image frames of the plurality of image frames; and
select at least one image frame of the one or more image frames based on metadata that corresponds to the at least one image frame.

### Claim 6 (depends on 5)

The system of claim 5, wherein the metadata that corresponds to the at least one image frame indicates at least one of:
a timestamp associated with the one or more points of time; or
that the at least one image frame was captured from the particular zone within the building.

### Claim 7 (depends on 1)

The system of claim 1, wherein generation of the plurality of image signatures includes the instructions causing the one or more processors to:
implement, prior to storage of the plurality of image signatures in the database, a data cache to store image signatures as they are output by the first machine learning model; and
as the image signatures are stored in the data cache:
compare one or more first image signatures of the image signatures with one or more second image signatures of the image signatures, wherein the one or more first image signatures correspond to one or more first image frames which precede one or more second image frames for which the one or more second image signatures correspond to; and
select, based at least on the one or more first image signatures or the one or more second image signatures describing the one or more first image frames and the one or more second image frames, at least one image signature from the one or more first image signatures or the one or more second image signatures to represent both the one or more first image signatures and the one or more second image signatures within the database.

### Claim 8 (depends on 7)

The system of claim 7, wherein comparison of the one or more first image signatures with the one or more second image signatures includes the instructions causing the one or more processors to:
determine that a vector difference between the one or more first image signatures and the one or more second image signatures is less than a threshold; and
detect, based on the vector difference being less than the threshold, that the at least one image signature describes both the one or more first image signatures and the one or more second image signatures.

### Claim 9 (depends on 7)

The system of claim 7, wherein comparison of the one or more first image signatures with the one or more second image signatures includes the instructions causing the one or more processors to:
determine that an amount of time elapsed between the one or more first image frames and the one or more second image frames is less than a predetermined threshold; and
determine, based on the amount of time being less than the predetermined threshold, whether the at least one image signature describes both the one or more first image frames and the one or more second image frames.

### Claim 10 (depends on 1)

The system of claim 1, wherein the first machine learning model includes an image encoder configured to generate one or more first vector embeddings, wherein the second machine learning model includes a text encoder configured to generate one or more second vector embeddings, and wherein performance of the search includes the instructions causing the one or more processors to:
perform a comparison between the one or more first vector embeddings and the one or more second vector embeddings.

### Claim 11 (depends on 10)

The system of claim 10, wherein the comparison between the one or more first vector embeddings and the one or more second vector embeddings includes the one or more processors to determine a cosine similarity between the one or more first vector embeddings and the one or more second vector embeddings.

### Claim 12

A method, comprising:
receiving, by one or more processing circuits, from one or more cameras of a building, image data;
extracting, by the one or more processing circuits, a plurality of image frames from the image data;
generating, by the one or more processing circuits, using a first machine learning model, a plurality of image signatures that describe features within the plurality of image frames;
storing, by the one or more processing circuits, responsive to generating the plurality of image signatures, the plurality of image signatures in a database;
receiving, by the one or more processing circuits, from a user device, a natural language query;
generating, by the one or more processing circuits, using a second machine learning model, a textual signature that describes the natural language query; and
performing, by the one or more processing circuits, responsive to generating the textual signature, a search of the database for one or more matches between the textual signature and the plurality of image signatures.

### Claim 13 (depends on 12)

The method of claim 12, further comprising:
detecting, by the one or more processing circuits, the one or more matches between the textual signature and at least one image signature of the plurality of image signatures;
identifying, by the one or more processing circuits, at least one image frame of the plurality of image frames that is described by the at least one image signature; and
outputting, by the one or more processing circuits, for display by a display device, the at least one image frame.

### Claim 14 (depends on 12)

The method of claim 12, wherein the first machine learning model includes an image encoder, wherein the second machine learning model includes a text encoder, and further comprising:
training, by the one or more processing circuits, the image encoder and the text encoder by:
compiling, by the one or more processing circuits, one or more sets of training data that include (i) one or more textual inputs and (ii) one or more image inputs, wherein the one or more textual inputs and the one or more image inputs both describe one or more training image frames;
providing, by the one or more processing circuits, the one or more textual inputs to the text encoder to cause the text encoder to generate one or more textual signatures;
providing, by the one or more processing circuits, the one or more image inputs to the image encoder to cause the image encoder to generate one or more image signatures;
detecting, by the one or more processing circuits, based on a comparison between the one or more textual signatures and the one or more image signatures, that an amount of variance between the image encoder and the text encoder adheres to one or more aspects of contrastive loss-based training; and
deploying, by the one or more processing circuits, the first machine learning model and the second machine learning model.

### Claim 15 (depends on 12)

The method of claim 12, further comprising:
while the first machine learning model is being trained:
providing, by the one or more processing circuits, to the second machine learning model, one or more textual inputs that provide a textual description of at least one image frame, wherein the one or more textual inputs cause the second machine learning model to output one or more textual signatures that describe the at least one image frame;
providing, by the one or more processing circuits, to the first machine learning model, the at least one image frame to cause the first machine learning model to output one or more image signatures that describe the at least one image frame; and
determining, by the one or more processing circuits, a performance of the first machine learning model based on a difference between the one or more textual signatures and the one or more image signatures.

### Claim 16

One or more non-transitory storage media storing instructions thereon that, when executed by one or more processors, cause the one or more processors to perform operations comprising:
receiving, from one or more cameras of a building, image data;
extracting a plurality of image frames from the image data;
generating, using a first machine learning model, a plurality of image signatures that describe features within the plurality of image frames;
storing, responsive to generating the plurality of image signatures, the plurality of image signatures in a database;
receiving, from a user device, a natural language query;
generating, using a second machine learning model, a textual signature that describes the natural language query; and
performing, responsive to generating the textual signature, a search of the database for one or more matches between the textual signature and the plurality of image signatures.

### Claim 17 (depends on 16)

The one or more non-transitory storage media of claim 16, wherein the operations further comprise:
detecting the one or more matches between the textual signature and at least one image signature of the plurality of image signatures;
identifying at least one image frame of the plurality of image frames that is described by the at least one image signature; and
outputting, for display by a display device, the at least one image frame.

### Claim 18 (depends on 16)

The one or more non-transitory storage media of claim 16, wherein the first machine learning model includes an image encoder, wherein the second machine learning model includes a text encoder, and wherein the operations further include:
training the image encoder and the text encoder by:
compiling one or more sets of training data that include (i) one or more textual inputs and (ii) one or more image inputs, wherein the one or more textual inputs and the one or more image inputs both describe one or more training image frames;
providing the one or more textual inputs to the text encoder to cause the text encoder to generate one or more textual signatures;
providing the one or more image inputs to the image encoder to cause the image encoder to generate one or more image signatures;
detecting, based on a comparison between the one or more textual signatures and the one or more image signatures, that an amount of variance between the image encoder and the text encoder adheres to one or more aspects of contrastive loss-based training; and
deploying the first machine learning model and the second machine learning model.

### Claim 19 (depends on 16)

The one or more non-transitory storage media of claim 16, wherein the operations further comprise:
while the first machine learning model is being trained:
providing, to the second machine learning model, one or more textual inputs that provide a textual description of at least one image frame, wherein the one or more textual inputs cause the second machine learning model to output one or more textual signatures that describe the at least one image frame;
providing, to the first machine learning model, the at least one image frame to cause the first machine learning model to output one or more image signatures that describe the at least one image frame; and
determining a performance of the first machine learning model based on a difference between the one or more textual signatures and the one or more image signatures.

### Claim 20 (depends on 16)

The one or more non-transitory storage media of claim 16, wherein the operations further comprise:
detecting that the natural language query includes an indication of one or more points in time or a particular zone within the building;
identifying, based on the search of the database, the one or more matches, wherein the one or more matches are between the textual signature and one or more image frames of the plurality of image frames; and
selecting at least one image frame of the one or more image frames based on metadata that corresponds to the at least one image frame.