Method for operating a mobile terminal using a gesture recognition and control device, gesture recognition and control device, motor vehicle, and an output apparatus that can be worn on the head

A gesture recognition and control device, comprising:

• a memory configured to store instructions;
• and
• a processor configured to execute the instructions stored in the memory to:
• recognize a mobile terminal,
• ascertain a graphical user interface as a GUI of the mobile terminal generated to be displayed by a display device of the mobile terminal, the GUI providing an operating option that involves an operating function of the mobile terminal being triggerable,
• provide an output signal describing the GUI of the mobile terminal as a display content of the mobile terminal,
• mirroring the display content of the mobile terminal in form of a virtual display content of the mobile terminal in an output region in an interior of a motor vehicle as part of an augmented reality or a virtual reality, output by a wearable output apparatus, based on the output signal describing the GUI of the mobile terminal as the display content of the mobile terminal,
• while the virtual display content of the mobile terminal is mirrored in the output region, recognize a contactless operating gesture of a user based on the operating option of the GUI of the mobile terminal,
• generate a remote control signal based on the contactless operating gesture, the remote control signal describing triggering of the operating function of the mobile terminal assigned to the contactless operating gesture, and
• transmit the remote control signal to a control device of the mobile terminal to trigger the operating function of the mobile terminal.

An output apparatus wearable on a head of a user for outputting augmented reality and/or virtual reality, the output apparatus comprising:

• a display;
• and
• a gesture recognition and control device, including:
• a memory configured to store instructions, and
• a processor configured to execute the instructions stored in the memory to:
• recognize a mobile terminal,
• ascertain a graphical user interface as a GUI of the mobile terminal generated to be displayed by a display device of the mobile terminal, the GUI providing an operating option that involves an operating function of the mobile terminal being triggerable,
• generate an output signal describing the GUI of mobile terminal as a display content of the mobile terminal,
• mirror the display content of the mobile terminal in form of a virtual display content of the mobile terminal in an output region in an interior of a motor vehicle as part of an augmented reality or a virtual reality, output by the output apparatus, based on the output signal describing the GUI of the mobile terminal as the display content of the mobile terminal,
• while the virtual display content of the mobile terminal is mirrored in the output region, recognize a contactless operating gesture of user based on the operating option of the GUI of the mobile terminal,
• generate a remote control signal based on the contactless operating gesture, the remote control signal describing triggering of the operating function of the mobile terminal assigned to the contactless operating gesture, and
• transmit the remote control signal to a control device of the mobile terminal to trigger the operating function of the mobile terminal.