US 11,886,489 B2Granted

System and method of identifying visual objects

AssigneeGoogle Inc.

InventorsDavid Petrou, Matthew Bridges, Shailesh Nalawadi, Hartwig Adam, Matthew R. Casey, Hartmut Neven, Andrew Harp

Life of this patent

Estimating the term…

Loading patent…

Abstract

A system and method of identifying objects is provided. In one aspect, the system and method includes a hand-held device with a display, camera and processor. As the camera captures images and displays them on the display, the processor compares the information retrieved in connection with one image with information retrieved in connection with subsequent images. The processor uses the result of such comparison to determine the object that is likely to be of greatest interest to the user. The display simultaneously displays the images the images as they are captured, the location of the object in an image, and information retrieved for the object.

Claims20

Dependent claims

A computer-implemented method, comprising:

displaying, by a user computing device comprising one or more processors, a video stream from a camera device associated with the user computing device, wherein the video stream comprises a plurality of image frames, and wherein the plurality of image frames collectively depict a plurality of objects;
providing, by the user computing device, one or more image frames of the plurality of image frames to an object recognition system;
in response to providing the one or more image frames, obtaining, by the user computing device from the object recognition system, annotation data descriptive of an object of the plurality of objects;
concurrently displaying, by the user computing device, at least one image frame of the plurality of image frames and the annotation data descriptive of the object;
receiving, by the user computing device from a user of the user computing device, a touch input associated with display of the annotation data descriptive of the object;
and
responsive to receiving the touch input, displaying, by the user computing device, additional annotation data descriptive of the object.

4 more elements

A user computing device comprising:

one or more processors;
and
one or more non-transitory computer-readable media comprising instructions that when executed by the one or more processors cause the one or more processors to perform operations comprising:
displaying a video stream from a camera device associated with the user computing device, wherein the video stream comprises a plurality of image frames, and wherein the plurality of image frames collectively depict a plurality of objects;
providing one or more image frames of the plurality of image frames to an object recognition system;
in response to providing the one or more image frames, obtaining, from the object recognition system, annotation data descriptive of an object of the plurality of objects;
concurrently displaying at least one image frame of the plurality of image frames and the annotation data descriptive of the object;
receiving, from a user of the user computing device, a touch input associated with display of the annotation data descriptive of the object;
and
responsive to receiving the touch input, displaying additional annotation data descriptive of the object.

7 more elements

Dependencies are inferred from the claim text.

Facts

Application: 18/189,776Filed 24 Mar 2023 · continuation
Priority: 6 Dec 2011
Published: US 2023/0237090 A127 Jul 2023
Granted: 30 Jan 2024
Term ends: Not estimatedThe source records don’t support an estimate.

Classification19 codes

G06F 16/535
Querying: filtering based on additional data, e.g. user or group profiles
G06F 3/048
Input arrangements or combined input and output arrangements for interaction between user and computer: interaction techniques based on graphical…
G06F 16/50
Information retrieval; Database structures therefor; File system structures therefor: of still image data
G06F 16/5838
Using metadata automatically derived from the content: using colour

Family

Application61/567,611Us provisional application · filed 6 Dec 2011
Application13/693,665Parent of this continuation · filed 4 Dec 2012
Application14/541,437Parent of this continuation · filed 14 Nov 2014
US 8,891,907 B2Parent of this continuation · filed 18 Nov 2014
Application15/247,542Parent of this continuation · filed 25 Aug 2016
US 9,442,957 B2Parent of this continuation · filed 13 Sep 2016
Application16/243,660Parent of this continuation · filed 9 Jan 2019
US 10,198,457 B2Parent of this continuation · filed 5 Feb 2019
Application16/563,375Parent of this continuation · filed 6 Sep 2019
US 10,409,855 B2Parent of this continuation · filed 10 Sep 2019
Application16/744,998Parent of this continuation · filed 16 Jan 2020
US 10,552,476 B2Parent of this continuation · filed 4 Feb 2020
Application17/157,022Parent of this continuation · filed 25 Jan 2021
US 10,902,055 B2Parent of this continuation · filed 26 Jan 2021
US 2023/0237090 A1Pre-grant publication · published 27 Jul 2023
US 11,615,136 B2Parent of this continuation
US 2013/0188886 A1Published application
US 2015/0112972 A1Published application
US 2016/0364414 A1Published application
US 2019/0146993 A1Published application
US 2019/0391996 A1Published application
US 2020/0151211 A1Published application
US 2021/0141827 A1Published application
US 11,886,489 B2This patent

From the US filing history. Foreign family members aren’t listed.