Imaging method for modular mixed reality (MR) device

An imaging method for modular mixed reality (MR) device, wherein

• the MR device comprising:
• an MR calculation module, an MR optical path module, and an MR posture module;
• the MR calculation module comprises a display assembly;
• the MR posture module comprises an imaging assembly and an IMU assembly, and the imaging assembly is configured to collect images in a preset angular direction of the display assembly, and the IMU assembly is configured to collect posture data of the MR device;
• the MR calculation module is connected to the MR posture module, adjusting display content of the display assembly according to the image data and posture data acquired by the MR posture module;
• the MR optical path module comprises a virtual-image optical path and a mixed optical path, the virtual-image optical path being connected to the display assembly;
• the input end of the mixed optical path is connected to the virtual-image optical path, while the output end of the mixed optical path is an observation end;
• a semi-transparent semi-reflective mirror is configured in the mixed optical path;
• one surface of the semi-transparent semi-reflective mirror is configured as a real-image introduction surface and the other surface of the semi-transparent semi-reflective mirror is configured as a virtual-image introduction surface;
• the real-image introduction surface faces a real environment, while the virtual-image introduction surface faces the virtual-image optical path;
• display content of the display assembly is processed and transmitted by means of the virtual-image optical path to form a virtual image;
• virtual-image light is reflected by means of the virtual-image introduction surface onto the observation end, and real environment light is transmitted to the observation end by means of the real-image introduction surface, being mixed with the virtual image to form a mixed reality image, the MR calculation module is a main smart phone, the display assembly is a display module of the main smart phone;
• the IMU assembly comprises a magnetometer, a gyroscope and an accelerometer;
• the IMU assembly has a main IMU assembly and an auxiliary IMU assembly;
• the main IMU assembly collects the posture data of the display assembly;
• the main IMU assembly is arranged in the main smart phone;
• the auxiliary IMU assembly is arranged in one or more control devices wirelessly connected to the main smart phone;
• the auxiliary IMU assembly collects the posture data or position data of the one or more control devices;
• the posture data includes posture angle, angular rate or acceleration data;
• the imaging assembly comprises a main imaging assembly and an auxiliary imaging assembly;
• the main imaging assembly is a rear camera of the main smart phone, and the auxiliary imaging assembly is a camera arranged at the control device,
• the semi-transparent semi-reflective mirror is configured to
• receive, when the MR optical path module is worn by a user, the virtual-image light from a direction above a horizontal line of sight of the user, and
• reflect the virtual-image light toward the observation end,
• the MR optical path module is a passive MR head-mounted mechanism;
• the main smart phone is fixed at the passive MR head-mounted mechanism;
• the control device is either a game console handle, a wearable device that can be worn on the hand or the foot, a sensor and a control device that are fixed at the passive MR head-mounted mechanism, or an auxiliary phone that is held by the user or is tied to the limbs of the user,
• the virtual-image optical path of the passive MR head-mounted mechanism comprises a resting plate, a total reflection mirror and a field lens;
• the field lens is combined by two Fresnel lenses;
• the main smart phone is placed horizontally on the resting plate;
• the main smart phone displays the VR split-screen mode image in the form of a horizontal double-split screen when the passive MR head-mounted mechanism is in operation;
• the image light of the horizontal double-split screen is reflected by means of the total reflection mirror onto the two Fresnel lenses, which then refract image light of the horizontal double-split screen, so that image light forms two virtual-image light paths with preset field angle,
• the orientation of the rear camera of the main smart phone is the orientation of the passive MR head-mounted mechanism;
• the posture data of the display assembly is the posture data of the main smart phone;
• the IMU assembly at the main smart phone collects the posture data of the main smart phone;
• when the passive MR head-mounted mechanism is in operation, the rear camera of the main smart phone collects feature points of the real scene at the initial orientation of the passive MR head-mounted mechanism, and successively collects images as posture images while the passive MR head-mounted mechanism is in operation;
• the MR calculation module adjusts the images on the horizontal double-split screen according to the variation of the feature points at the posture image and the variation of the posture data of the main smart phone,
• the passive MR head-mounted mechanism is made up of a sheet, which is provided with an A-folding section, a B-folding section and a C-folding section along the length;
• the A-folding section is fixed with the semi-transparent and semi-reflective mirror and the field lens;
• the B-folding section is fixed with the total reflective mirror;
• the C-folding section is provided with resting plate;
• the resting plate is provided with an observation hole for collecting external images by the rear camera of the main smart phone;
• the method for preparing the passive MR head- mounted mechanism comprises the steps of:
• B1, folding the A-folding section and the B-folding section to a rhombic column, so that the lens is located at the connection line of the rhombic vertex;
• one side surface of the rhombic column is open and is an incident surface of image light, and the other three side surfaces are closed and form an observation hole wall, a semi-transparent semi-reflective mirror wall and a total reflective mirror wall respectively;
• the incident surface of image light faces the total reflective mirror wall;
• the total reflective mirror wall is provided with the total reflective mirror;
• the observation hole locates at the observation hole wall;
• the side wall of the rhombic column facing the observation hole is the semi-transparent semi-reflective mirror;
• the semi-transparent semi-reflective mirror locates at the semi-transparent semi-reflective mirror wall;
• B2, expanding a light shield at the A-folding section and inserting the light shield into the observation hole wall;
• and
• B3, expanding the C-folding section and putting the main smart phone having rear camera on the resting plate, so that the rear camera aligns to the observation hole of the resting plate;
• folding the C-folding section then to the incident surface of light image of the rhombic column;
• the observation end includes the observation hole, at which the mixed reality image formed by mixing the screen image of the phone with the external image can be seen when the main smart phone displays the VR split screen mode image in the form of the horizontal double-split screen.