Multi-dimensional data capture of an environment using plural devices
AssigneeAemass, Inc.
InventorMarshall Reed Millett
Life of this patent
Estimating the term…
Abstract
Embodiments of the invention describe apparatuses, systems, and methods related to data capture of objects and/or an environment. In one embodiment, a user can capture time-indexed three-dimensional (3D) depth data using one or more portable data capture devices that can capture time indexed color images of a scene with depth information and location and orientation data. In addition, the data capture devices may be configured to captured a spherical view of the environment around the data capture device.
Claims17
A method of delivering three-dimensional (3D) data to two or more users, the method comprising:
- generating, by a computer processing system, a series of time-indexed 3D frames that combines 3D frames of image data of at least two scenes in motion captured by two or more sets of data capture devices over a first period of time, wherein each of the at least two scenes in motion captures 3D frames of image data associated with different users of the two or more users, and wherein the series of time-indexed 3D frames for a first scene in motion associated with a first user and a second scene in motion associated with a second user are generated by:
- receiving 3D frames of image data, from each data capture device of a first set of data capture devices associated with the first user and a second set of data capture devices associated with the second user, each data capture device including a depth sensor that captures a frame of depth data having an array of depth measurements corresponding to an array of pixels of image data captured by said each data capture device, the 3D frames of image data from said each data capture device including an array of pixels of image data of at least one of the first or second scene in motion and time-indexed depth data frames indicating an array of distance measurements corresponding to the array of pixels of image data to points on surfaces of the at least one of the first second or second scene in motion sampled over the first period of time from one or more points of view,
- receiving location data for the first period of time, the location data including data for a location of each of said each data capture device, determining 3D coordinates in a common coordinate system for each of the points of the depth data and corresponding pixels of image data using the location data and the array of distance measurements captured by said each data capture device at each sampled time of the first period of time, and
- generating a first series of time-indexed 3D frames of the first scene in motion associated with the first user, wherein each of the time-indexed 3D frames of the first scene in motion includes a 3D point cloud with the 3D coordinates of the points of the depth data for each data capture device from the first set of data capture devices at a given sampled time of the first period of time;
- generating a second series of time-indexed 3D frames of the second scene in motion associated with the second user, wherein each of the time-indexed 3D frames of the second scene in motion includes a 3D point cloud with the 3D coordinates of the points of the depth data for each data capture device from the second set of data capture devices at the given sampled time of the first period of time;
- and
- transmitting, by the computer processing system, (i) the first series of time-indexed 3D frames to a second user computer system associated with the second user, (ii) the second series of time-indexed 3D frames to a first user computer system associated with the first user, for viewing of the first scene in motion by the second user at the second user computer system while viewing the second scene in motion by the first user at the first user computer system.
A system for delivering three-dimensional (3D) data to two or more users, the system comprising:
- a memory;
- a computer processing system coupled with the memory and configured to perform operations, comprising:
- generating a series of time-indexed 3D frames that combines 3D frames of image data of at least two scenes in motion captured by two or more sets of data capture devices over a first period of time, wherein each of the at least two scenes in motion captures 3D frames of image data associated with different users of the two or more users, and wherein the series of time-indexed 3D frames for a first scene in motion associated with a first user and a second scene in motion associated with a second user are generated by:
- receiving 3D frames of image data, from each data capture device of a first set of data capture devices associated with the first user and a second set of data capture devices associated with the second user, each data capture device including a depth sensor that captures a frame of depth data having an array of depth measurements corresponding to an array of pixels of image data captured by said each data capture device, the 3D frames of image data from said each data capture device including an array of pixels of image data of at least one of the first or second scene in motion and time-indexed depth data frames indicating an array of distance measurements corresponding to the array of pixels of image data to points on surfaces of the at least one of the first second or second scene in motion sampled over the first period of time from one or more points of view,
- receiving location data for the first period of time, the location data including data for a location of each of said each data capture device,
- determining 3D coordinates in a common coordinate system for each of the points of the depth data and corresponding pixels of image data using the location data and the array of distance measurements captured by said each data capture device at each sampled time of the first period of time, and
- generating a first series of time-indexed 3D frames of the first scene in motion associated with the first user, wherein each of the time-indexed 3D frames of the first scene in motion includes a 3D point cloud with the 3D coordinates of the points of the depth data for each data capture device from the first set of data capture devices at a given sampled time of the first period of time;
- generating a second series of time-indexed 3D frames of the second scene in motion associated with the second user, wherein each of the time-indexed 3D frames of the second scene in motion includes a 3D point cloud with the 3D coordinates of the points of the depth data for each data capture device from the second set of data capture devices at the given sampled time of the first period of time;
- and
- a transceiver, coupled with the computer processing system, and configured to perform operations comprising:
- transmitting (i) the first series of time-indexed 3D frames to a second user computer system associated with the second user, (ii) the second series of time-indexed 3D frames to a first user computer system associated with the first user, for viewing of the first scene in motion by the second user at the second user computer system while viewing the second scene in motion by the first user at the first user computer system.
A non-transitory computer-readable storage medium having stored thereon data representing sequences of instructions that, when executed by a computer processing system, cause the computer processing system to perform operations for delivering three-dimensional (3D) data to two or more users, the operations comprising:
- generating a series of time-indexed 3D frames that combines 3D frames of image data of at least two scenes in motion captured by two or more sets of data capture devices over a first period of time, wherein each of the at least two scenes in motion captures 3D frames of image data associated with different users of the two or more users, and wherein the series of time-indexed 3D frames for a first scene in motion associated with a first user and a second scene in motion associated with a second user are generated by:
- receiving 3D frames of image data, from each data capture device of a first set of data capture devices associated with the first user and a second set of data capture devices associated with the second user, each data capture device including a depth sensor that captures a frame of depth data having an array of depth measurements corresponding to an array of pixels of image data captured by said each data capture device, the 3D frames of image data from said each data capture device including an array of pixels of image data of at least one of the first or second scene in motion and time-indexed depth data frames indicating an array of distance measurements corresponding to the array of pixels of image data to points on surfaces of the at least one of the first second or second scene in motion sampled over the first period of time from one or more points of view,
- receiving location data for the first period of time, the location data including data for a location of each of said each data capture device,
- determining 3D coordinates in a common coordinate system for each of the points of the depth data and corresponding pixels of image data using the location data and the array of distance measurements captured by said each data capture device at each sampled time of the first period of time, and
- generating a first series of time-indexed 3D frames of the first scene in motion associated with the first user, wherein each of the time-indexed 3D frames of the first scene in motion includes a 3D point cloud with the 3D coordinates of the points of the depth data for each data capture device from the first set of data capture devices at a given sampled time of the first period of time;
- generating a second series of time-indexed 3D frames of the second scene in motion associated with the second user, wherein each of the time-indexed 3D frames of the second scene in motion includes a 3D point cloud with the 3D coordinates of the points of the depth data for each data capture device from the second set of data capture devices at the given sampled time of the first period of time;
- and
- transmitting (i) the first series of time-indexed 3D frames to a second user computer system associated with the second user, (ii) the second series of time-indexed 3D frames to a first user computer system associated with the first user, for viewing of the first scene in motion by the second user at the second user computer system while viewing the second scene in motion by the first user at the first user computer system.
Dependencies are inferred from the claim text.