Image processing device, image processing method, and image provision system for the display of viewpoint images of a shared environment

An image processing method, comprising:

• receiving each of:
• a first viewpoint image of a first user, wherein the first viewpoint image is viewable from a viewpoint of the first user in a virtual space,
• a second viewpoint image of a second user, wherein the second viewpoint image is viewable from a viewpoint of the second user in the virtual space,
• viewpoint position information that indicates a position of the second user, and
• line-of-sight information indicating a line-of-sight direction of the second user;
• controlling superimposition of the second viewpoint image of the second user on the first viewpoint image of the first user based on a positional relationship of the first user and the second user in the virtual space, wherein the positional relationship is based on the viewpoint position information and the line-of-sight information of the second user;
• controlling display of each of the first viewpoint image, and the second viewpoint image, wherein the second viewpoint image is superimposed on the first viewpoint image as a sub-screen on a display screen of the first user;
• and
• controlling change of at least one of a position of the displayed second viewpoint image, a size of the displayed second viewpoint image, or transparency of the displayed second viewpoint image based on line-of-sight information of the first user, wherein the line-of-sight information of the first user indicates a line-of-sight direction of the first user with respect to the sub-screen.

An image provision system, comprising:

• an image processing device that includes a central processing unit (CPU) configured to:
• receive each of:
• a first viewpoint image of a first user, wherein the first viewpoint image is viewable from a viewpoint of the first user in a virtual space,
• a second viewpoint image of a second user, wherein the second viewpoint image is viewable from a viewpoint of the second user in the virtual space,
• viewpoint position information that indicates a position of the second user, and
• line-of-sight information indicating a line-of-sight direction of the second user;
• control superimposition of the second viewpoint image of the second user on the first viewpoint image of the first user based on a positional relationship of the first user and the second user in the virtual space, wherein the positional relationship is based on the viewpoint position information and the line-of-sight information of the second user;
• control display of each of the first viewpoint image, and the second viewpoint image, wherein the second viewpoint image is superimposed on the first viewpoint image as a sub-screen on a display screen of the first user;
• and
• control change of at least one of a position of the displayed second viewpoint image, a size of the displayed second viewpoint image, or transparency of the displayed second viewpoint image based on line-of-sight information of the first user, wherein the line-of-sight information of the first user indicates a line-of-sight direction of the first user with respect to the sub-screen;
• and
• an image provision device that includes circuitry configured to transmit the first viewpoint image and the second viewpoint image to the image processing device at bit rate distribution based on the size or the transparency of the second viewpoint image.

An image processing device, comprising:

• a central processing unit (CPU) configured to:
• receive each of:
• a first viewpoint image of a first user, wherein the first viewpoint image is viewable from a viewpoint of the first user in a virtual space,
• a second viewpoint image of a second user, wherein the second viewpoint image is viewable from a viewpoint of the second user in the virtual space,
• viewpoint position information that indicates a position of the second user, and
• line-of-sight information indicating a line-of-sight direction of the second user;
• control superimposition of the second viewpoint image of the second user on the first viewpoint image of the first user based on a positional relationship of the first user and the second user in the virtual space,
• wherein the positional relationship is based on the viewpoint position information and the line-of-sight information of the second user;
• control display of each of the first viewpoint image, and the second viewpoint image superimposed on the first viewpoint image;
• and
• obtain specific information from line-of-sight information of the first user and the line-of-sight information of the second user, wherein the specific information indicates a ratio at which a visual field of the first user overlaps with a visual field of the second user;
• control increase in at least one of a size or a transmittance of the displayed second viewpoint image in a case where the ratio increases;
• reduce at least one of the size or the transmittance of the displayed second viewpoint image in a case where the ratio decreases;
• and
• control end of the display of the second viewpoint image in a case where the ratio is one of equal to or larger than a threshold.