Image processing method, recording medium, and image processing system

A non-transitory computer readable recording medium to cause a computer to execute the image processing method comprising:

• estimating a structure of a space inside a construction from a background image in which the space is imaged in all directions;
• detecting a subject appearing in the background image and a three-dimensional (3D) position of an imaging device which captured the background image, the subject including a fixed structural component of the construction;
• estimating a 3D position of the subject in the space;
• estimating, based on the structure of the space, the 3D position of the subject, information of the subject and information of a virtual object to be placed:
• an arrangement allowable region, in which the virtual object is allowed to be arranged in the space, by generating a 3D space model based on the estimated structure of the space, calculating distances between potential placement positions and detected subject positions in the 3D space model, and identifying positions that satisfy predetermined distance thresholds from the 3D position of the subject and from other objects as the arrangement allowable region;
• an arrangement prohibited region, in which the virtual object is prohibited from being arranged in the space, by defining a 3D coordinate region in the 3D space model that includes the 3D position of the imaging device, expanding the 3D coordinate region to include a predetermined radius around the imaging device position, and designating the expanded coordinate region as unavailable for virtual object placement;
• and
• combining the virtual object with the background image in the arrangement allowable region, and not in the arrangement prohibited region, to generate a staged image including the virtual object in the space.

An image processing system, comprising:

• circuitry configured to
• estimate a structure of a space inside a construction from a background image in which the space is imaged in all directions;
• detect a subject appearing in the background image and a three-dimensional (3D) position of an imaging device which captured the background image, the subject including a fixed structural component of the construction;
• estimate a 3D position of the subject in the space;
• estimate, based on the structure of the space, the 3D position of the subject, information of the subject and information of a virtual object to be placed:
• an arrangement allowable region, in which the virtual object is allowed to be arranged in the space, by generating a 3D space model based on the estimated structure of the space, calculating distances between potential placement positions and detected subject positions in the 3D space model, and identifying positions that satisfy predetermined distance thresholds from the 3D position of the subject and from other objects as the arrangement allowable region;
• an arrangement prohibited region, in which the virtual object is prohibited from being arranged in the space, by defining a 3D coordinate region in the 3D space model that includes the 3D position of the imaging device, expanding the 3D coordinate region to include a predetermined radius around the imaging device position, and designating the expanded coordinate region as unavailable for virtual object placement;
• and
• combine the virtual object with the background image in the arrangement allowable region, and not in the arrangement prohibited region, to generate a staged image including the virtual object in the space.