Utilizing object attribute detection models to automatically select instances of detected objects in images
AssigneeAdobe Inc.
InventorsScott Cohen, Zhe Lin, Mingyang Ling
Life of this patent
Estimating the term…
Abstract
The present disclosure relates to an object selection system that accurately detects and automatically selects user-requested objects (e.g., query objects) in a digital image. For example, the object selection system builds and utilizes an object selection pipeline to determine which object detection neural network to utilize to detect a query object based on analyzing the object class of the query object. In addition, the object selection system can add, update, or replace portions of the object selection pipeline to improve overall accuracy and efficiency of automatic object selection within an image.
Claims20
A non-transitory computer-readable medium storing instructions that, when executed by at least one processor, cause a computing device to:
- determine, from a selection query, a query object to be selected from a plurality of objects in a digital image;
- determine an object class or object type by analyzing the query object from the selection query;
- select, based on the object class or object type of the query object, to utilize a first object detection neural network from a plurality of object detection neural networks, the plurality of object detection neural networks comprising a specialist object detection neural network, a concept-based object detection neural network, a known object class detection neural network, and an unknown object class detection neural network;
- detect the query object in the digital image utilizing the first object detection neural network;
- generate, based on the detected query object in the digital image, a first object mask of the detected query object utilizing an object mask neural network;
- and
- provide the image with the first object mask in response to the selection query.
A system for automatically selecting objects within digital images comprising:
- a memory comprising:
- a digital image;
- a plurality of object detection neural networks comprising a specialist object detection neural network, a concept-based object detection neural network, a known object class detection neural network, a regional proposal neural network, and a concept embedding neural network;
- and
- an object mask neural network;
- at least one processor;
- and
- at least one non-transitory computer-readable storage medium storing instructions that, when executed by the at least one processor, cause the system to:
- determine, from a selection query, a noun identifying a query object to be selected within the digital image;
- analyze the noun to determine an object class type of the query object;
- determine to utilize a first object detection neural network from the plurality of object detection neural networks based on correlating the object class type of the query object to the first object detection neural network;
- detect the query object and generate an approximate boundary for the query object utilizing the first object detection neural network;
- generate a first object mask of the detected query object based on the object mask neural network and the approximate boundary of the detected query object;
- and
- provide the digital image with the first object mask in response to the selection query.
In a digital medium environment for creating or editing digital images, a computer-implemented method of selecting query objects, comprising:
- identifying a query string that identifies a query object in a digital image;
- a step for detecting the query object utilizing an object detection neural network selected from a plurality of object detection neural networks;
- generating a first object mask of the detected query object utilizing an object mask neural network;
- and
- providing the digital image with the first object mask to a client device associated with a user.
Dependencies are inferred from the claim text.