# US 10,769,849 B2 · Use of temporal motion vectors for 3D reconstruction > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,769,849 B2 - Document ID: US10769849B2 - Status: Granted - Kind: B2 - Application number: 15/772,740 - Filed: 2016-11-04 - Published: 2019-07-11 - Granted: 2020-09-08 - Earliest claimed: 2015-11-04 - Assignees: Intel Corporation - Applicants: Intel Corporation - Inventors: Sridhar Uyyala; Bradley A. Jackson; Ignacio J. Alvarez; Deepak S. Vembar - Web page: https://patentgrep.com/patent/US10769849B2 ## CPC - G06T 17/205: {Re-meshing} - G06T 7/215: Motion-based segmentation - G06T 7/292: Multi-camera tracking - G06T 7/55: from multiple images - G06T 15/005: {General purpose rendering architectures} - G06T 15/08: Volume rendering - G06T 17/005: {Tree description, e.g. octree, quadtree} - G06T 2207/20072: Graph-based image processing - G06T 2207/20081: Training; Learning - G06T 2210/08: Bandwidth reduction - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Techniques for three-dimensional (3D) reconstruction of a dynamic scene as a set of voxels are provided. One technique includes: receiving, by a processor, image data from each of two or more spatially-separated sensors observing the scene from a corresponding two or more vantage points; fusing, by the processor, the image data into the set of voxels on a frame-by-frame basis; segmenting, by the processor, the image data into objects that constitute the scene; detecting, by the processor, which of the objects remain static from frame to frame, remaining ones of the objects being dynamic; filtering, by the processor, the set of voxels to remove those of the voxels corresponding to the static objects, to produce a dynamic subset of the voxels; and outputting, by the processor to a display device, those of the voxels corresponding to the dynamic objects (such as the dynamic subset) and not to the static objects. ## Claims ### Claim 1 A method for three-dimensional (3D) reconstruction of a dynamic scene as a set of voxels, the method comprising: receiving, by a processor-based system, image data from each of two or more spatially-separated sensors observing the scene from a corresponding two or more vantage points; fusing, by the processor-based system, the image data into a first set of voxels on a frame-by-frame basis; segmenting, by the processor-based system, the image data into objects that constitute the scene; detecting, by the processor-based system, which of the objects remain static from frame to frame, remaining ones of the objects being dynamic; mapping the static objects to their corresponding voxels of the first set; removing the voxels corresponding to the static objects from the first set of voxels to form a second set of voxels different from the first set of voxels, the second set including only dynamic voxels; and outputting, by the processor-based system to a display device, the second set of voxels and not any voxels corresponding to the static objects from the first set. ### Claim 2 (depends on 1) The method of claim 1, further comprising selecting, by the processor-based system, only those of the voxels currently visible to a user of the display device, to produce a sparse subset of the voxels, wherein the outputting of the second set of voxels further comprises only outputting those of the second set of voxels belonging to the sparse subset. ### Claim 3 (depends on 2) The method of claim 2, further comprising organizing, by the processor-based system, the sparse subset in a sparse voxel octree (SVO) format. ### Claim 4 (depends on 1) The method of claim 1, wherein the display device is a 3D display device or an SVO ray cast display device. ### Claim 5 (depends on 1) The method of claim 1, wherein the segmenting comprises segmenting the image data into objects using machine learning algorithms. ### Claim 6 (depends on 1) The method of claim 1, wherein the receiving comprises receiving image data from each of three or more equidistantly separated sensors. ### Claim 7 A system for three-dimensional (3D) reconstruction of a dynamic scene as a set of voxels, the system comprising: a data capture circuit configured to receive image data from each of two or more spatially-separated sensors observing the scene from a corresponding two or more vantage points; an image processing circuit configured to fuse the image data into a first set of voxels on a frame-by-frame basis; an object segmentation circuit configured to segment the image data into objects that constitute the scene; a motion detection circuit configured to detect which of the objects remain static from frame to frame, remaining ones of the objects being dynamic; a voxel mapping circuit configured to map the static objects to their corresponding ones of the voxels of the first set; a static filter circuit configured to remove the voxels corresponding to the static objects from the first set of voxels to produce a second set of voxels different from the first set of voxels, the second set including only dynamic voxels; and a voxel output circuit configured to output, to a display device, the second set of voxels and not any voxels corresponding to the static objects from the first set. ### Claim 8 (depends on 7) The system of claim 7, further comprising a voxel selection circuit configured to select only those of the voxels currently visible to a user of the display device, to produce a sparse subset of the voxels, wherein the voxel output circuit is further to only output those of the second set of voxels belonging to the sparse subset. ### Claim 9 (depends on 8) The system of claim 8, wherein the voxel selection circuit is further configured to organize the sparse subset in a sparse voxel octree (SVO) format. ### Claim 10 (depends on 8) The system of claim 8, wherein the display device is a 3D display device or an SVO ray cast display device. ### Claim 11 (depends on 8) The system of claim 8, wherein the object segmentation circuit is configured to segment the image data into objects using machine learning algorithms. ### Claim 12 (depends on 8) The system of claim 8, wherein the two or more spatially-separated sensors comprise three or more equidistantly separated sensors. ### Claim 13 At least one non-transitory computer readable storage medium having instructions encoded thereon that, when executed by one or more processors, result in the following operations for three-dimensional (3D) reconstruction of a dynamic scene as a set of voxels being performed, the operations comprising: receiving image data from each of two or more spatially-separated sensors observing the scene from a corresponding two or more vantage points; fusing the image data into a first set of voxels on a frame-by-frame basis; segmenting the image data into objects that constitute the scene; detecting which of the objects remain static from frame to frame, remaining ones of the objects being dynamic; mapping the static objects to their corresponding voxels of the first set; removing the voxels corresponding to the static objects from the first set of voxels to form a second set of voxels different from the first set of voxels, the second set including only dynamic voxels; and outputting to a display device the second set of voxels and not any voxels corresponding to the static objects from the first set. ### Claim 14 (depends on 13) The computer readable storage medium of claim 13, the operations further comprising selecting only those of the voxels currently visible to a user of the display device, to produce a sparse subset of the voxels, wherein the outputting of the second set of voxels further comprises only outputting those of the second set of voxels belonging to the sparse subset. ### Claim 15 (depends on 14) The computer readable storage medium of claim 14, the operations further comprising organizing the sparse subset in a sparse voxel octree (SVO) format. ### Claim 16 (depends on 13) The computer readable storage medium of claim 13, wherein the display device is a 3D display device or an SVO ray cast display device. ### Claim 17 (depends on 13) The computer readable storage medium of claim 13, wherein the segmenting comprises segmenting the image data into objects using machine learning algorithms. ### Claim 18 (depends on 13) The computer readable storage medium of claim 13, wherein the receiving comprises receiving image data from each of three or more equidistantly separated sensors.