Back to results
US 9,426,451 B2Granted
Cooperative photography
AssigneeDigimarc Corporation
InventorGeoffrey B. Rhoads
Life of this patent
Estimating the term…
Abstract
Imagery from two or more users' different smartphones is streamed to a cloud processor, enabling creation of 3D model information about a scene being imaged. From this model, arbitrary views and streams can be synthesized. In one arrangement, a user of such a system is at a sports arena, and her view of the sporting event is blocked when another spectator rises to his feet in front of her. Nonetheless, the imagery presented on her headworn display continues uninterrupted—the blocked imagery from that viewpoint being seamlessly re-created based on imagery contributed by other system users in the arena. A great variety of other features and arrangements are also detailed.
Claims11
A method comprising the acts:
- from a first device conveyed by a first user having a first viewpoint of a scene, receiving first data, the first data including (a) a first stream of captured time-changing imagery depicting said scene over a time interval, (b) first scene depth information identifying distances from the first device to plural points in the scene, and (c) time-based synchronization information, a first view of the scene from said first viewpoint being momentarily interrupted by a first moving occlusion between the first device and the scene;
- and
- providing said first user of the first device a view of the scene, without said interrupting first moving occlusion, from said first viewpoint, said providing employing a hardware processor to synthesize imagery to replace occluded imagery, using other data from one or more other devices conveyed by one or more other users having one or more other viewpoints of said scene different than said first viewpoint, said other data including, for each of said one or more other devices:
- (a) another stream of captured time-changing imagery depicting said scene over a time interval, (b) other scene depth information identifying distances from the other device to plural points in the scene, and (c) time-based synchronization information, said other data being used by said hardware processor to synthesize said imagery;
- wherein a transient scene occlusion in the stream of imagery captured by the first user's device is mitigated based on data from one or more streams of imagery captured by one or more other users' devices.
2 more elements
A system including an interface, a processor and a memory, the interface being adapted to exchange data with first, second and third camera-equipped user devices, the memory including programming instructions that configure the system to perform acts including:
- receiving first data, the first data including first scene image data captured by the first device camera and depicting a view of a subject from a first viewpoint with a transient interrupting occlusion, the first data also including first scene depth map information indicating distance from the first device camera to each of plural points in the first scene image data;
- receiving second data, the second data including second scene image data captured by the second device camera and depicting a view of the subject from a second viewpoint without the interrupting occlusion, the second data also including second scene depth map information indicating distance from the second device camera to each of plural points in the second scene image data;
- receiving third data, the third data including third scene image data captured by the third device camera and depicting a view of the subject from a third viewpoint without the interrupting occlusion, the third data also including third scene depth map information indicating distance from the third device camera to each of plural points in the third scene image data;
- generating projected first disk data corresponding to positions and energies of plural first disks projected onto the first scene imaged by the first device camera, said generating being based on (a) a point spread function for the first device camera, (b) said first scene depth map information, and (c) said first viewpoint;
- generating projected second disk data corresponding to positions and energies of plural second disks projected onto the second scene imaged by the second device camera, said generating being based on (a) a point spread function for the second device camera, (b) said second scene depth map information, and (c) said second viewpoint;
- generating projected third disk data corresponding to positions and energies of plural third disks projected onto the third scene imaged by the third device camera, said generating being based on (a) a point spread function for the third device camera, (b) said third scene depth map information, and (c) said third viewpoint;
- synthesizing an image of the subject without said interrupting occlusion using said projected first, second and third disk data;
- wherein a transient scene occlusion in the imagery captured by the first device is mitigated based on data from streams of imagery captured by the second and third devices.
5 more elements
A method comprising the acts:
- receiving first data, the first data including first scene image data captured by a first camera depicting a view of a subject from a first viewpoint with an interrupting occlusion, the first data also including first scene depth map information indicating distance from the first camera to each of plural points in the first scene image data;
- receiving second data, the second data including second scene image data captured by a second camera depicting a view of the subject from a second viewpoint without the interrupting occlusion, the second data also including second scene depth map information indicating distance from the second camera to each of plural points in the second scene image data;
- generating projected first disk data corresponding to positions and energies of plural first disks projected onto the first scene imaged by the first camera, said generating being based on (a) a point spread function for the first camera, (b) said first scene depth map information, and (c) said first viewpoint;
- generating projected second disk data corresponding to positions and energies of plural second disks projected onto the second scene imaged by the second camera, said generating being based on (a) a point spread function for the second camera, (b) said second scene depth map information, and (c) said second viewpoint;
- and
- based on both said projected first and second disk data, synthesizing an image of the subject without said interrupting occlusion.
3 more elements
Dependencies are inferred from the claim text.