Image processing method for augmented reality device, augmented reality device and storage medium

An augmented reality device, comprising:

• an environment image capturing camera module, configured to capture an environment image;
• a human eye image capturing camera module, configured to capture a human eye image;
• a memory configured to store executable instructions;
• and
• a processor, configured to implement following steps when executing the executable instructions stored in the memory:
• tracking lines of sight of a left eye and a right eye to determine a position of a binocular watching range in a three-dimensional space;
• acquiring an image of an external environment where the augmented reality (AR) device is located, and acquiring environment map point cloud data based on a Simultaneous Localization and Mapping (SLAM) algorithm;
• and
• updating the environment map point cloud data within the binocular watching range in response to detecting an update of the image of the external environment;
• wherein the tracking the lines of sight of the left eye and the right eye to determine the position of the binocular watching range in the three-dimensional space comprises:
• capturing an image of a left eye and an image of a right eye, determining a center position of a left pupil according to the image of the left eye, and determining a center position of a right pupil according to the image of the right eye;
• inputting pupil center position data of both eyes into a sight watching model to determine a position of an intersection point of the lines of sight of both eyes on a virtual imaging plane;
• and
• determining a midpoint of a line connecting a left exit pupil center and a right exit pupil center of a binocular optical imaging module of the AR device, constructing a sight cone by taking the line connecting the midpoint and the intersection point as a central axis, and taking a space within the sight cone as the binocular watching range;
• wherein a vertex of the sight cone is the midpoint;
• wherein the sight watching model is used to describe a mapping relationship between coordinates of the centers of the pupils of both eyes in a coordinate system and coordinates of watching points of the lines of sight of both eyes on left and right imaging planes of the binocular optical imaging module of the AR device in the coordinate system.

A non-transitory computer readable storage medium storing executable instructions, wherein when the executable instructions are executed by a processor, the following steps are implemented:

• tracking lines of sight of a left eye and a right eye to determine a position of a binocular watching range in a three-dimensional space;
• acquiring an image of an external environment where the augmented reality (AR) device is located, and acquiring environment map point cloud data based on a Simultaneous Localization and Mapping (SLAM) algorithm;
• and
• updating the environment map point cloud data within the binocular watching range in response to detecting an update of the image of the external environment;
• wherein the tracking the lines of sight of the left eye and the right eye to determine the position of the binocular watching range in the three-dimensional space comprises:
• capturing an image of a left eye and an image of a right eye, determining a center position of a left pupil according to the image of the left eye, and determining a center position of a right pupil according to the image of the right eye;
• inputting pupil center position data of both eyes into a sight watching model to determine a position of an intersection point of the lines of sight of both eyes on a virtual imaging plane;
• and
• determining a midpoint of a line connecting a left exit pupil center and a right exit pupil center of a binocular optical imaging module of the AR device, constructing a sight cone by taking the line connecting the midpoint and the intersection point as a central axis, and taking a space within the sight cone as the binocular watching range;
• wherein a vertex of the sight cone is the midpoint.