US 10,853,911 B2Granted

Dynamic adaptation of images for projection, and/or of projection parameters, based on user(s) in environment

AssigneeGoogle Inc.

InventorJian Wei Leong

Life of this patent

Estimating the term…

Loading patent…

Abstract

Implementations relate to dynamic adaptation of images for projection by a projector, based on one or more properties of user(s) that are in an environment with the projector. The projector can be associated with an automated assistant client of a client device. In some versions of those implementations, a pose of a user in the environment is determined and, based on the pose, a base image for projecting onto a surface is warped to generate a transformed image. The transformed image, when projected onto a surface and viewed from the pose of the user, mitigates perceived differences relative to the base image. The base image (on which the transformed image is based) can optionally be generated in dependence on a distance of the user. Some implementations additionally or alternatively relate to dynamic adaptation of projection parameters (e.g., a location for projection, a size of projection) based on one or more properties of user(s) that are in an environment with the projector.

Claims19

Dependent claims

A method implemented by one or more processors, comprising:

identifying, by an automated assistant client of a computing device in an environment, a base image for projecting onto a surface via a projector accessible to the automated assistant client;
determining, using sensor data from at least one sensor, a first pose of a user in the environment, wherein the sensor data is accessible to the automated assistant client, wherein identifying the base image comprises selecting the base image, from a plurality of base images, based on the first pose of the user;
determining, using the first pose of the user, first image transformation parameters for warping images;
generating a first transformed image that is a linear transformation of the base image, wherein generating the first transformed image comprises using the first image transformation parameters to warp the base image, wherein the first transformed image, when projected onto the surface and viewed from the first pose of the user, mitigates perceived differences relative to the base image;
causing the projector to project the first transformed image onto the surface;
determining, using additional sensor data from the at least one sensor, a second pose of the user in the environment, wherein the second pose of the user indicates the user has moved;
determining, using the second pose of the user, second image transformation parameters for warping images;
generating a second transformed image that is a linear transformation of the base image or of an additional base image, wherein generating the second transformed image comprises using the second transformation parameters to warp the base image or the additional base image, wherein the second transformed image, when projected onto the surface and viewed from the second pose of the user, mitigates perceived differences relative to the base image or the additional base image;
and
causing the projector to project the second transformed image onto the surface.

7 more elements

A method implemented by one or more processors, comprising:

identifying, by an automated assistant client of a computing device, a base image for projecting onto a surface via a projector accessible to the automated assistant client;
identifying a plurality of users that are in an environment with the computing device;
determining, using sensor data from at least one sensor accessible to the computing device, that a subset of the users are active users for the automated assistant client;
determining at least one pose for the subset of the users that are determined to be active users, wherein determining the at least one pose is based on the sensor data or additional sensor data from at least one additional sensor accessible to the computing device;
based on determining that the subset of the users are active users, using the at least one pose for the subset of the users in generating a transformed image of the base image;
and
causing the projector to project the transformed image onto the surface.

4 more elements

Dependencies are inferred from the claim text.

Facts

Application: 15/955,297Filed 17 Apr 2018
Published: US 2019/0325556 A124 Oct 2019
Granted: 1 Dec 2020
Term ends: Not estimatedThe source records don’t support an estimate.

Classification22 codes

H04N 9/3185
Video signal processing therefor: geometric adjustment, e.g. keystone or convergence
G06F 3/011
Input arrangements or combined input and output arrangements for interaction between user and computer: arrangements for interaction with the…
G06F 40/20
Handling natural language data: natural language analysis
G06T 3/00
Geometric image transformations in the plane of the image