System and method for head mount display removal processing

An information processing apparatus comprising

• one or more memories storing instructions;
• and
• one or more processors that, upon execution of the stored instructions, are configured to cause the one or more processors to:
• receive at least two consecutive image frames captured live by an image capture apparatus, each of the at least two image frames including a subject wearing a head mount display device;
• generate a first bounding box surrounding the head mount display device using orientation information obtained from the head mount display;
• generate a second bounding box surrounding the head mount display using an object detection model trained to identify the head mount display;
• calculate a difference indicator by comparing differences, on a pixel by pixel basis between the generated first and second bounding boxes;
• select the first bounding box when it is determined that the difference indicator exceeds a predetermined threshold;
• provide coordinates representing the first bounding box to identify a region within the images to be replaced by a region of a precaptured image that is occluded by the head mount display device.

A system comprising:

• a head mount display device configured to be worn by a subject;
• an image capture device configured to capture real time images of the subject wearing the head mount display device;
• and
• an information processing apparatus one or more memories storing instructions;
• and one or more processors that, upon execution of the stored instructions, are configured to cause the one or more processors to:
• receive at least two consecutive image frames captured live by an image capture apparatus, each of the at least two image frames including a subject wearing a head mount display device;
• generate a first bounding box surrounding the head mount display device using orientation information obtained from the head mount display;
• generate a second bounding box surrounding the head mount display using an object detection model trained to identify the head mount display;
• calculate a difference indicator by comparing differences, on a pixel by pixel basis between the generated first and second bounding boxes;
• select the first bounding box when it is determined that the difference indicator exceeds a predetermined threshold;
• provide coordinates representing the first bounding box to identify a region within the images to be replaced by a region of a precaptured image that is occluded by the head mount display device.