US 12,614,357 B2 · Apparatus and methods for augmenting vision with region-of-interest based processing

Abstract

Systems, apparatus, and methods for augmenting vision with region-of-interest based processing. In one specific example, smart glasses may use an eye-tracking camera to monitor the user's gaze and determine the user's gaze point. When triggered, the camera assembly captures a high-resolution image. The high-resolution image may be cropped to a much smaller region-of-interest (ROI) image based on computer-vision analysis of the user's gaze point. For example, if the smart glasses detect a human face at the gaze point, then the ROI is cropped to the human face. In this manner, the smart glasses may leverage specific capabilities of the smart glasses to augment the user experience; for example, telephoto lenses provide long distance vision, or computer-assisted search may direct the user to interesting activity. Other aspects may include e.g., external database assisted operation and/or ongoing cataloging throughout the day.

Claims20

Dependent claims

A smart glasses apparatus, comprising:

a physical frame;
an outward-facing camera assembly;
a display;
a processor;
and
a non-transitory computer-readable medium comprising instructions that, when executed by the processor, cause the smart glasses apparatus to:
receive faceprint data from a companion device based on a user calendar entry;
capture a first image data via the outward-facing camera assembly;
determine a region-of-interest based on the first image data;
crop the region-of-interest from the first image data forming a cropped image;
attempt object recognition on the cropped image using the faceprint data;
connect to the companion device, in response to unsuccessfully recognizing an object in the cropped image;
transmit the cropped image to the companion device;
instruct the companion device to perform object recognition on the cropped image;
obtain supplemental data about the object from the companion device based on the cropped image;
and
display the supplemental data via the display.

14 more elements

A smart glasses apparatus, comprising:

a physical frame;
an outward-facing camera assembly;
a display;
a processor;
and
a non-transitory computer-readable medium comprising instructions that, when executed by the processor, cause the smart glasses apparatus to:
capture a first image data via the outward-facing camera assembly;
attempt object recognition on the first image data based on faceprint data stored on the smart glasses apparatus;
connect to a companion device, in response to unsuccessfully recognizing an object in the first image data;
transmit at least a portion of the first image data to the companion device;
instruct the companion device to perform object recognition on the at least the portion of the first image data;
obtain supplemental data about the object from the companion device based on transmitting at least the portion of the first image data and instructing the companion device;
and
display the supplemental data via the display.

11 more elements

Dependencies are inferred from the claim text.

Apparatus and methods for augmenting vision with region-of-interest based processing

Life of this patent

Apparatus and methods for augmenting vision with region-of-interest based processing

Life of this patent

Abstract

Claims20