Back to results
US 12,008,696 B2Granted
Translation method and AR device
AssigneeBeijing Xiaomi Mobile Software Co., Ltd.
InventorsZhixiong Yu, Zifei Dou, Xiang Li
Life of this patent
Estimating the term…
Abstract
A translation method is performed by an augmented reality (AR) device. The method includes obtaining a text to be translated from an environment image in response to a translation trigger instruction; obtaining a translation result of the text to be translated; and displaying the translation result based on a preset display mode in an AR space. The AR space is a virtual space constructed by the AR device based on a real environment.
Claims17
A translation method, performed by an augmented reality (AR) device, comprising:
- obtaining a text to be translated from an environment image in response to a trigger instruction;
- obtaining a translation result of the text to be translated;
- and
- displaying the translation result based on a preset display mode in an AR space, wherein the AR space is a virtual space constructed by the AR device based on a real environment;
- wherein obtaining the text to be translated from the environment image in response to the trigger instruction comprises:
- recognizing a user gesture in the environment image;
- determining a target text image pointed by the user gesture;
- generating the trigger instruction based on the target text image;
- and
- determining the target text image as an image of the text to be translated based on the trigger instruction;
- wherein determining the target text image pointed by the user gesture comprises:
- recognizing a direction of the user gesture in the environment image and at least one first text image in the environment image;
- determining a text image of the at least one first text image along the recognized direction as the target text image;
- or
- collecting a new environment image based on the user gesture;
- recognizing a direction of the user gesture in the new environment image and at least one second text image in the new environment image;
- determining a text image of the at least one second text image along the recognized direction as the target text image.
14 more elements
An augmented reality (AR) device, comprising:
- a frame,
- a camera arranged on the frame, configured to collect an environment image,
- a microphone arranged on the frame, configured to receive a trigger instruction sent by a user, and
- a memory and a processor arranged on the frame, the memory being configured to store instructions executable by a processor, the processor being configured to:
- obtain a text to be translated from the environment image in response to a trigger instruction;
- obtain a translation result of the text to be translated and control display of the translation result based on a preset display mode in an AR space, wherein the AR space is a virtual space constructed by the AR device based on a real environment;
- wherein the processor is further configured to:
- recognize a user gesture in the environment image;
- determine a target text image pointed by the user gesture;
- generate the trigger instruction based on the target text image;
- and
- determine the target text image as an image of the text to be translated based on the trigger instruction;
- wherein the processor is further configured to perform:
- recognizing a direction of the user gesture in the environment image and at least one first text image in the environment image;
- determining a text image of the at least one first text image along the recognized direction as the target text image;
- or
- collecting a new environment image based on the user gesture;
- recognizing a direction of the user gesture in the new environment image and at least one second text image in the new environment image;
- determining a text image of the at least one second text image along the recognized direction as the target text image.
16 more elements
A non-transitory computer readable storage medium having a computer program stored thereon, wherein when the computer program is performed by a processor, a translation method performed by an augmented reality (AR) device, the method comprising:
- obtaining a text to be translated from an environment image in response to a trigger instruction;
- obtaining a translation result of the text to be translated;
- and
- displaying the translation result based on a preset display mode in an AR space, wherein the AR space is a virtual space constructed by the AR device based on a real environment;
- wherein obtaining the text to be translated from the environment image in response to the trigger instruction comprises:
- recognizing a user gesture in the environment image;
- determining a target text image pointed by the user gesture;
- generating the trigger instruction based on the target text image;
- and
- determining the target text image as an image of the text to be translated based on the trigger instruction;
- wherein determining the target text image pointed by the user gesture comprises:
- recognizing a direction of the user gesture in the environment image and at least one first text image in the environment image;
- determining a text image of the at least one first text image along the recognized direction as the target text image;
- or
- collecting a new environment image based on the user gesture;
- recognizing a direction of the user gesture in the new environment image and at least one second text image in the new environment image;
- determining a text image of the at least one second text image along the recognized direction as the target text image.
14 more elements
Dependencies are inferred from the claim text.