Point cloud data communication system, point cloud data transmitting apparatus, and point cloud data transmission method

A point cloud data communication system, comprising:

• a transmitting apparatus;
• and
• a receiving apparatus,
• wherein point cloud data is transmitted from the transmitting apparatus to the receiving apparatus via a communication network,
• the transmitting apparatus includes
• a point cloud data storage unit that stores point cloud data indicating a stereoscopic shape in a three-dimensional space,
• a display request receiving unit that receives a display request transmitted from the receiving apparatus,
• a point cloud data extracting unit that extracts point cloud data of a part corresponding to the display request received by the display request receiving unit from among the point cloud data stored in the point cloud data storage unit,
• a thinning-out unit that decides a part with a low priority in accordance with a display state of the stereographic shape specified on the basis of a designation included in the display request, and extracts point cloud data of the part with the low priority from among the point cloud data extracted by the point cloud data extracting unit, and
• a point cloud data transmitting unit that transmits the point cloud data remaining after thinning-out by the thinning-out unit among the point cloud data extracted by the point cloud data extracting unit to the receiving apparatus, and
• the receiving apparatus includes
• a display request transmitting unit that transmits the display request including designation related to a display state of the stereographic shape in the three-dimensional space to the transmitting apparatus, and
• a point cloud data receiving unit that receives the point cloud data transmitted from the transmitting apparatus in response to the display request transmitted by the display request transmitting unit,
• wherein the point cloud data stored in the point cloud data storage unit is point cloud data indicating a stereoscopic image in which the stereoscopic shapes included in a plurality of captured images are synthesized into one, the stereoscopic image being generated by setting a plurality of projection planes in accordance with a relative angular relation of a plurality of cameras installed to photograph a photographing target area from a plurality of angles, calculating depth information indicating distances of the stereoscopic shapes included in the plurality of captured images captured by the plurality of cameras from the cameras for each pixel, and projecting values of pixels of the stereoscopic shapes included in the plurality of captured images in directions of the projection planes corresponding to the captured images in accordance with the depth information,
• at least one of a viewpoint and a direction of a line of sight from the viewpoint is variable in the three-dimensional space,
• the display request transmitting unit transmits the display request including at least one of the viewpoint and the direction of the line of sight designated by a user as the designation related to the display state of the stereoscopic shape to the transmitting apparatus,
• the point cloud data extracting unit extracts point cloud data of a part to which a priority is given when the stereoscopic shape is displayed in the three-dimensional space in accordance with at least one of the viewpoint and the direction of the line of sight included in the display request from among the point cloud data stored in the point cloud data storage unit, and
• the thinning-out unit calculates, when a part is thinned out from among the point cloud data extracted by the point cloud data extracting unit, a distance from the viewpoint to the stereoscopic shape on the basis of position information of each point projected in accordance with the depth information, and increases a thinning-out amount of point cloud data of a part related to a stereoscopic shape having a long distance from the viewpoint.

A point cloud data transmitting apparatus that transmits point cloud data to a receiving apparatus via a communication network, comprising:

• a point cloud data storage unit that stores point cloud data indicating a stereoscopic shape in a three-dimensional space;
• a display request receiving unit that receives a display request including designation related to a display state of the stereoscopic shape in the three-dimensional space from the receiving apparatus;
• a point cloud data extracting unit that extracts point cloud data of a part corresponding to the display request received by the display request receiving unit from among the point cloud data stored in the point cloud data storage unit;
• and
• a point cloud data transmitting unit that transmits the point cloud data extracted by the point cloud data extracting unit to the receiving apparatus,
• wherein the point cloud data stored in the point cloud data storage unit is point cloud data indicating a stereoscopic image in which the stereoscopic shapes included in a plurality of captured images are synthesized into one, the stereoscopic image being generated by setting a plurality of projection planes in accordance with a relative angular relation of a plurality of cameras installed to photograph a photographing target area from a plurality of angles, calculating depth information indicating distances of the stereoscopic shapes included in the plurality of captured images captured by the plurality of cameras from the cameras for each pixel, and projecting values of pixels of the stereoscopic shapes included in the plurality of captured images in directions of the projection planes corresponding to the captured images in accordance with the depth information,
• at least one of a viewpoint and a direction of a line of sight from the viewpoint is variable in the three-dimensional space,
• the display request receiving unit receives the display request including at least one of the viewpoint and the direction of the line of sight designated by a user as the designation related to the display state of the stereoscopic shape from the receiving apparatus, and
• the point cloud data extracting unit decides a camera of a direction to which a priority is given in accordance with the viewpoint on the basis of a relation between the viewpoint included in the display request and positions of the plurality of cameras, and extracts the point cloud data generated from the values of the pixels included in a captured image by the decided camera from among the point cloud data stored in the point cloud data storage unit as point cloud data of a part to which a priority is given when the stereographic shape is displayed in the three-dimensional space.

A method of transmitting point cloud data indicating a stereoscopic shape in a three-dimensional space from a transmitting apparatus including a point cloud data storage unit that stores the point cloud data to a receiving apparatus via a communication network, comprising:

• a step of receiving, by a display request receiving unit of the transmitting apparatus, a display request including designation related to a display state of the stereoscopic shape in the three-dimensional space from the receiving apparatus;
• a step of extracting, by a point cloud data extracting unit of the transmitting apparatus, point cloud data of a part corresponding to the display request received by the display request receiving unit from among the point cloud data stored in the point cloud data storage unit;
• and
• a step of transmitting, by the point cloud data transmitting unit of the transmitting apparatus, the point cloud data extracted by the point cloud data extracting unit to the receiving apparatus,
• wherein the point cloud data stored in the point cloud data storage unit is point cloud data indicating a stereoscopic image in which the stereoscopic shapes included in a plurality of captured images are synthesized into one, the stereoscopic image being generated by setting a plurality of projection planes in accordance with a relative angular relation of a plurality of cameras installed to photograph a photographing target area from a plurality of angles, calculating depth information indicating distances of the stereoscopic shapes included in the plurality of captured images captured by the plurality of cameras from the cameras for each pixel, and projecting values of pixels of the stereoscopic shapes included in the plurality of captured images in directions of the projection planes corresponding to the captured images in accordance with the depth information,
• at least one of a viewpoint and a direction of a line of sight from the viewpoint is variable in the three-dimensional space,
• the display request receiving unit receives the display request including at least one of the viewpoint and the direction of the line of sight designated by a user as the designation related to the display state of the stereoscopic shape from the receiving apparatus, and
• the point cloud data extracting unit decides a camera of a direction to which a priority is given in accordance with the viewpoint on the basis of a relation between the viewpoint included in the display request and positions of the plurality of cameras, and extracts the point cloud data generated from the values of the pixels included in a captured image by the decided camera from among the point cloud data stored in the point cloud data storage unit as point cloud data of a part to which a priority is given when the stereographic shape is displayed in the three-dimensional space.

A point cloud data transmitting apparatus that transmits point cloud data to a receiving apparatus via a communication network, comprising:

• a point cloud data storage unit that stores point cloud data indicating a stereoscopic shape in a three-dimensional space;
• a display request receiving unit that receives a display request including designation related to a display state of the stereoscopic shape in the three-dimensional space from the receiving apparatus;
• a point cloud data extracting unit that extracts point cloud data of a part corresponding to the display request received by the display request receiving unit from among the point cloud data stored in the point cloud data storage unit;
• a thinning-out unit that decides a part with a low priority in accordance with the display state of the stereographic shape specified on the basis of the designation included in the display request, and extracts point cloud data of the part with the low priority from among the point cloud data extracted by the point cloud data extracting unit;
• and
• a point cloud data transmitting unit that transmits the point cloud data remaining after thinning-out by the thinning-out unit among the point cloud data extracted by the point cloud data extracting unit to the receiving apparatus,
• wherein the point cloud data stored in the point cloud data storage unit is point cloud data indicating a stereoscopic image in which the stereoscopic shapes included in a plurality of captured images are synthesized into one, the stereoscopic image being generated by setting a plurality of projection planes in accordance with a relative angular relation of a plurality of cameras installed to photograph a photographing target area from a plurality of angles, calculating depth information indicating distances of the stereoscopic shapes included in the plurality of captured images captured by the plurality of cameras from the cameras for each pixel, and projecting values of pixels of the stereoscopic shapes included in the plurality of captured images in directions of the projection planes corresponding to the captured images in accordance with the depth information,
• at least one of a viewpoint and a direction of a line of sight from the viewpoint is variable in the three-dimensional space,
• the display request receiving unit receives the display request including at least one of the viewpoint and the direction of the line of sight designated by a user as the designation related to the display state of the stereoscopic shape from the receiving apparatus,
• the point cloud data extracting unit extracts point cloud data of a part to which a priority is given when the stereoscopic shape is displayed in the three-dimensional space in accordance with at least one of the viewpoint and the direction of the line of sight included in the display request from among the point cloud data stored in the point cloud data storage unit, and
• the thinning-out unit calculates, when a part is thinned out from among the point cloud data extracted by the point cloud data extracting unit, a distance from the viewpoint to the stereoscopic shape on the basis of position information of each point projected in accordance with the depth information, and increases a thinning-out amount of point cloud data of a part related to a stereoscopic shape having a long distance from the viewpoint.

A method of transmitting point cloud data indicating a stereoscopic shape in a three-dimensional space from a transmitting apparatus including a point cloud data storage unit that stores the point cloud data to a receiving apparatus via a communication network, comprising:

• a step of receiving, by a display request receiving unit of the transmitting apparatus, a display request including designation related to a display state of the stereoscopic shape in the three-dimensional space from the receiving apparatus;
• a step of extracting, by a point cloud data extracting unit of the transmitting apparatus, point cloud data of a part corresponding to the display request received by the display request receiving unit from among the point cloud data stored in the point cloud data storage unit;
• and
• a step of deciding, by a thinning-out unit of the transmitting apparatus, a part with a low priority in accordance with the display state of the stereographic shape specified on the basis of the designation included in the display request, and extracting point cloud data of the part with the low priority from among the point cloud data extracted by the point cloud data extracting unit;
• and
• a step of transmitting, by the point cloud data transmitting unit of the transmitting apparatus, the point cloud data remaining after thinning-out by the thinning-out unit among the point cloud data extracted by the point cloud data extracting unit to the receiving apparatus,
• wherein the point cloud data stored in the point cloud data storage unit is point cloud data indicating a stereoscopic image in which the stereoscopic shapes included in a plurality of captured images are synthesized into one, the stereoscopic image being generated by setting a plurality of projection planes in accordance with a relative angular relation of a plurality of cameras installed to photograph a photographing target area from a plurality of angles, calculating depth information indicating distances of the stereoscopic shapes included in the plurality of captured images captured by the plurality of cameras from the cameras for each pixel, and projecting values of pixels of the stereoscopic shapes included in the plurality of captured images in directions of the projection planes corresponding to the captured images in accordance with the depth information,
• at least one of a viewpoint and a direction of a line of sight from the viewpoint is variable in the three-dimensional space,
• the display request receiving unit receives the display request including at least one of the viewpoint and the direction of the line of sight designated by a user as the designation related to the display state of the stereoscopic shape from the receiving apparatus,
• the point cloud data extracting unit extracts point cloud data of a part to which a priority is given when the stereoscopic shape is displayed in the three-dimensional space in accordance with at least one of the viewpoint and the direction of the line of sight included in the display request from among the point cloud data stored in the point cloud data storage unit, and
• the thinning-out unit calculates, when a part is thinned out from among the point cloud data extracted by the point cloud data extracting unit, a distance from the viewpoint to the stereoscopic shape on the basis of position information of each point projected in accordance with the depth information, and increases a thinning-out amount of point cloud data of a part related to a stereoscopic shape having a long distance from the viewpoint.