Back to results
US 2024/0179291 A1Published application
Generating 3d video using 2d images and audio with background keyed to 2d image-derived metadata
AssigneeSony Interactive Entertainment Inc.
InventorDaisuke Kawamura
Life of this patent
Estimating the term…
Abstract
3D video is generated based on end user 2D photos and videos stored in a section of memory designated as 3D memory. Characters in the 3D video are based on the textures of people in the 2D images, and environmental background in the 3D video is selected from a model most closely resembling images in the 2D video. The 3D video is updated over time such that a user can select which period of a person's life to view in 3D
Claims20
An apparatus comprising:
- at least one processor configured to:
- access 2D image information in storage;
- using the 2D image information, generate 3D interactive video at least in part by matching at least one feature of at least one object in the 2D image information to at least one 3D model.
A device comprising:
- at least one computer storage that is not a transitory signal and that comprises instructions executable by at least one processor to:
- identify at least one texture of at least one human image in a 2D image;
- use the texture to generate a 3D character;
- identify at least one environmental background type in the 2D image;
- based at least in part on the environmental background type in the 2D image, select an environmental background type model;
- use the environmental background type model to generate a 3D environmental background;
- and
- merge the 3D environmental background with the 3D character to generate a 3D video.
5 more elements
A method, comprising:
- accessing 2D images in a 3D spatial memory;
- automatically creating a 3D video based on the 2D images at least in part using an environmental background model selected based on an environmental background in the 2D images as classified by a machine learning (ML) model and a texture of an image of a human face in the 2D images.
Dependencies are inferred from the claim text.