US 11,961,291 B2 · Visual search for electronic devices

Abstract

The subject technology provides visual search systems and methods that can be used to efficiently perform visual searches on an electronic device. The subject technology provides systems and methods for presenting one or more visual indicators corresponding to the searchable portions of digital content. A visual search may include identifying, at an electronic device, an element of interest in an image. A visual indicator for the element of interest may be overlaid on the image at a location corresponding to the element of interest. The visual indicator may be selectable to cause a display of information associated with the element of interest responsive to the selection.

Claims20

Dependent claims

A method, comprising:

obtaining, by a system process of a computing device, an image being displayed by an application at the computing device;
identifying, by the system process and responsive to providing the image to one or more machine learning models at the computing device, an element of interest in the image, the identifying further comprising:
identifying a plurality of elements in the image using the one or more machine learning models;
ranking the plurality of elements based on a ranking algorithm;
and
based on the ranking, determining the element of interest, from amongst the plurality of elements;
and
displaying a visual indicator for the element of interest overlaid on the image at a location corresponding to the element of interest, wherein the visual indicator is selectable to cause a display of information associated with the element of interest.

5 more elements

A system comprising:

a processor;
and
a memory device containing instructions, which when executed by the processor, cause the processor to:
obtain, by a system process of a computing device, an image being displayed by an application at the computing device;
identify, by the system process and responsive to providing the image to one or more machine learning models at the computing device, an element of interest in the image, the identify further comprising:
identify a plurality of elements in the image using the one or more machine learning models;
rank the plurality of elements based on a ranking algorithm;
and
based on the ranking, determine the element of interest, from amongst the plurality of elements;
and
display a visual indicator for the element of interest overlaid on the image at a location corresponding to the element of interest, wherein the visual indicator is selectable to cause a display of information associated with the element of interest.

8 more elements

A non-transitory machine-readable medium comprising instructions that, when executed by a processor, causes the processor to perform operations comprising:

obtaining, by a system process of a computing device, an image being displayed by an application at the computing device;
identifying, by the system process and responsive to providing the image to one or more machine learning model at the computing device, an element of interest in the image, the identifying further comprising:
identifying a plurality of elements in the image using the one or more machine learning models;
ranking the plurality of elements based on a ranking algorithm;
and
based on the ranking, determining the element of interest, from amongst the plurality of elements;
and
displaying a visual indicator for the element of interest overlaid on the image at a location corresponding to the element of interest, wherein the visual indicator is selectable to cause a display of information associated with the element of interest.

5 more elements

Dependencies are inferred from the claim text.

Visual search for electronic devices

Life of this patent

Visual search for electronic devices

Life of this patent

Abstract

Claims20