US 12,591,617 B1Granted

Neural network retraining based on image classification feedback

AssigneeMatroid, Inc.

InventorsHuaijin Wang, Alexander Jordan Bildner, Amey Patel, Andrew Ellison, John Goddard, Ryan Wong, Darin Tay, Reza Bosagh Zadeh

Life of this patent

Estimating the term…

patentgrep

Loading patent…

Abstract

Described herein are systems and methods that highlight target objects in media content. The detection system trains a neural network to identify objects within an image. The detection system receives a text input requesting a search within the image and applies a search language model to the text input, which identifies a target object associated with the requested search. The detection system applies the neural network to the image to identify instances of the target object. The detection system modifies a user interface to include the image and modifies the image to highlight the identified instances of the target object. The detection system receives feedback that modifies the highlighted instances of the target object within the user interface and retrains the neural network based on the modified highlighted instances.

Claims20

Dependent claims

A method comprising:

training, by a detection system, a neural network to identify objects within a first image;
receiving, at the detection system, a text input requesting a search within the first image;
applying, by the detection system, an LLM to the received text input to identify a target object associated with the requested search;
applying, by the detection system, the neural network to the first image to identify instances of the target object;
modifying, by the detection system, a first portion of a user interface to include the first image;
modifying, by the detection system, the first image within the first portion of the user interface to highlight the identified instances of the target object;
receiving, by the detection system, text indicative of one or more modifications to the highlighted instances of the target object;
applying, by the detection system, a second LLM to determine modification criteria for the one or more modifications;
generating, by the detection system, a second image based on application of the modification criteria to the first image;
and
retraining, by the detection system, the neural network based on the modified highlighted instances of the target object.

8 more elements

A non-transitory computer-readable storage medium storing instructions that, when executed, cause a processor to perform operations comprising:

training, by a detection system, a neural network to identify objects within a first image;
receiving, at the detection system, a text input requesting a search within the first image;
applying, by the detection system, an LLM to the received text input to identify a target object associated with the requested search;
applying, by the detection system, the neural network to the first image to identify instances of the target object;
modifying, by the detection system, a first portion of a user interface to include the first image;
modifying, by the detection system, the first image within the first portion of the user interface to highlight the identified instances of the target object;
receiving, by the detection system, text indicative of one or more modifications to the highlighted instances of the target object;
applying, by the detection system, a second LLM to determine modification criteria for the one or more modifications;
generating, by the detection system, a second image based on application of the modification criteria to the first image;
and
retraining, by the detection system, the neural network based on the modified highlighted instances of the target object.

8 more elements

A system comprising:

a processor;
and
a non-transitory computer-readable storage medium storing instructions that, when executed, cause the processor to perform operations comprising:
training, by a detection system, a neural network to identify objects within a first image;
receiving, at the detection system, a text input requesting a search within the first image;
applying, by the detection system, an LLM to the received text input to identify a target object associated with the requested search;
applying, by the detection system, the neural network to the first image to identify instances of the target object;
modifying, by the detection system, a first portion of a user interface to include the first image;
modifying, by the detection system, the first image within the first portion of the user interface to highlight the identified instances of the target object;
receiving, by the detection system, text indicative of one or more modifications to the highlighted instances of the target object;
applying, by the detection system, a second LLM to determine modification criteria for the one or more modifications;
generating, by the detection system, a second image based on application of the modification criteria to the first image;
and
retraining, by the detection system, the neural network based on the modified highlighted instances of the target object.

11 more elements

Dependencies are inferred from the claim text.