Method, device, and computer program product for room layout

An electronic device, comprising:

• a processor and a memory;
• the memory for storing computer execution instructions;
• the processor executing the computer execution instructions stored in the memory to cause the processor to implement a method for room layout comprising:
• collecting a current frame RGB image, and acquiring a depth map corresponding to the current frame RGB image and pose information of a head mounted device;
• detecting and determining at least one object in the collected current frame RGB image, wherein each of the at least one object corresponds to at least one bounding box, and each of the at least one object is of a plane type or a cuboid type;
• and
• associating the at least one object in the current frame RGB image, the depth image and the pose information with a room layout map corresponding to a previous frame RGB image, and generating a room layout map corresponding to the current frame RGB image for enabling a user to calibrate and create a rendering based on the room layout map corresponding to the current frame RGB image,
• wherein associating the at least one object in the current frame RGB image, the depth image and the pose information with the room layout map corresponding to the previous frame RGB image, and generating the room layout map corresponding to the current frame RGB image, comprise:
• for an object of the at least one object,
• in case that the object is of a plane type, determining a valid box corresponding to the object, and obtaining, based on the valid box, the depth map and the pose information, an updated first room layout map by associating the object with the room layout map corresponding to the previous frame RGB image;
• in case that the object is of a cuboid type, projecting, based on points of respective objects of the cuboid type in a world coordinate system in the room layout map corresponding to the previous frame RGB image, 3D bounding boxes of the respective objects of the cuboid type in the room layout map corresponding to the previous frame RGB image into the detected current frame RGB image, and obtaining, based on the bounding boxes of the respective objects of the cuboid type in the at least one object, the projected 3D bounding boxes and the depth map, an updated second room layout map by associating the object with the room layout map corresponding to the previous frame RGB image;
• and
• fusing the updated first room layout map and the updated second room layout map, to generate the room layout map corresponding to the current frame RGB image.

A non-transitory computer readable storage medium having computer execution instructions stored therein which, when executed by a processor, implements a method for room layout comprising:

• collecting a current frame RGB image, and acquiring a depth map corresponding to the current frame RGB image and pose information of a head mounted device;
• detecting and determining at least one object in the collected current frame RGB image, wherein each of the at least one object corresponds to at least one bounding box, and each of the at least one object is of a plane type or a cuboid type;
• and
• associating the at least one object in the current frame RGB image, the depth image and the pose information with a room layout map corresponding to a previous frame RGB image, and generating a room layout map corresponding to the current frame RGB image for enabling a user to calibrate and create a rendering based on the room layout map corresponding to the current frame RGB image,
• wherein associating the at least one object in the current frame RGB image, the depth image and the pose information with the room layout map corresponding to the previous frame RGB image, and generating the room layout map corresponding to the current frame RGB image, comprise:
• for an object of the at least one object,
• in case that the object is of a plane type, determining a valid box corresponding to the object, and obtaining, based on the valid box, the depth map and the pose information, an updated first room layout map by associating the object with the room layout map corresponding to the previous frame RGB image;
• in case that the object is of a cuboid type, projecting, based on points of respective objects of the cuboid type in a world coordinate system in the room layout map corresponding to the previous frame RGB image, 3D bounding boxes of the respective objects of the cuboid type in the room layout map corresponding to the previous frame RGB image into the detected current frame RGB image, and obtaining, based on the bounding boxes of the respective objects of the cuboid type in the at least one object, the projected 3D bounding boxes and the depth map, an updated second room layout map by associating the object with the room layout map corresponding to the previous frame RGB image;
• and
• fusing the updated first room layout map and the updated second room layout map, to generate the room layout map corresponding to the current frame RGB image.