US 11,200,650 B1Granted

Dynamic image re-timing

AssigneeApple Inc.

InventorsNir Ben Zadok, Oz Barak, Inna Stainvas, Moshe Laifenfeld, Andrei Kolin

Life of this patent

Estimating the term…

Loading patent…

Abstract

Techniques for the modification of at least part of a target image (e.g., scene objects within the target image), e.g., to make the target image appear that it was captured at a different time (e.g., a different time of day, different time of year) are disclosed. This “dynamic re-timing” of the target image may be achieved by finding one or more source images including the same (or similar) scene depicted in the target image (but, e.g., captured at different times), extracting stylistic elements from the source image(s), and then modifying at least part of the target image in a realistic fashion (e.g., not altering the geometry of objects in the target image), based on one or more extracted stylistic elements from the source image(s). Three-dimensional modeling of scene objects may allow a more realistic-looking transfer of the extracted stylistic elements onto scene objects in the target image to be achieved.

Claims20

Dependent claims

An image processing method, comprising:

obtaining a first image of a first scene at a first capture time;
obtaining location information related to the first image;
comparing the first image to a first plurality of images stored in an image database, wherein each of the first plurality of images is associated with location information and a capture time;
identifying a first subset of images from the first plurality of images based, at least in part, on the respective location information of the first subset of images;
selecting a second image from the first subset of images based, at least in part, on the second image's associated location information, wherein the second image depicts at least a part of the first scene, and wherein the capture time of the second image is different than the first capture time;
identifying at least a first structural element and at least a first stylistic element characterizing the second image;
extracting the at least first stylistic element from the second image, wherein the at least first stylistic element from the second image is based, at least in part, on the capture time of the second image;
and
applying the extracted at least first stylistic element to at least part of the first image to create a stylized version of the first image;
and
storing the stylized version of the first image in a memory.

8 more elements

A non-transitory program storage device comprising instructions stored thereon to cause one or more processors to:

obtain a first image of a first scene at a first capture time;
obtain location information related to the first image;
compare the first image to a first plurality of images stored in an image database, wherein each of the first plurality of images is associated with location information and a capture time;
identify a first subset of images from the first plurality of images based, at least in part, on the respective location information of the first subset of images;
select a second image from the first subset of images based, at least in part, on the second image's associated location information, wherein the second image depicts at least a part of the first scene, and wherein the capture time of the second image is different than the first capture time;
determine a first scene object that is present in each of the first image and the second image;
identify at least a first structural element and at least a first stylistic element characterizing the second image;
extract the at least first stylistic element from the second image, wherein the at least first stylistic element from the second image is based, at least in part, on the capture time of the second image;
and
apply the extracted at least first stylistic element to at least the part of the first image to create a stylized version of the first image;
and
store the stylized version of the first image in a memory.

9 more elements

A device, comprising:

a memory;
one or more camera units;
a display;
a user interface;
and
one or more processors operatively coupled to the memory, wherein the one or more processors are configured to execute instructions causing the one or more processors to:
obtain, from the one or more camera units, a first image of a first scene at a first capture time;
obtain location information related to the first image;
compare the first image to a first plurality of images stored in an image database, wherein each of the first plurality of images is associated with location information and a capture time;
identify a first subset of images from the first plurality of images based, at least in part, on the respective location information of the first subset of images;
select a second image from the first subset of images based, at least in part, on the second image's associated location information, wherein the second image depicts at least a part of the first scene, and wherein the capture time of the second image is different than the first capture time;
identify at least a first structural element and at least a first stylistic element characterizing the second image;
extract the at least first stylistic element from the second image, wherein the at least first stylistic element from the second image is based, at least in part, on the capture time of the second image;
and
apply the extracted at least first stylistic element to at least part of the first image to create a stylized version of the first image;
and
store the stylized version of the first image in the memory.

14 more elements

Dependencies are inferred from the claim text.

Facts

Application: 16/578,991Filed 23 Sep 2019 · us provisional application
Priority: 26 Sep 2018
Granted: 14 Dec 2021
Term ends: Not estimatedThe source records don’t support an estimate.

Classification17 codes

G06F 16/532
Querying: query formulation, e.g. graphical querying
G06F 16/55
Of still image data: clustering; Classification
G06F 16/5854
Using metadata automatically derived from the content: using shape and object relationship
G06F 18/2115
Selection of the most significant subset of features: by evaluating different subsets according to an optimisation criterion, e.g. class…