Information processing apparatus that determines whether a user can visually recognize a mobile terminal in a mixed reality image, system, control method, and non-transitory computer storage medium

A control method of a computer, the control method comprising:

• generating an image to be displayed on a display device that is mountable to the head of a user;
• deciding an operation mode of an application;
• accepting an input operation from a mobile terminal that the user is capable of holding and operating, and performing an application operation on the basis of the operation mode and the input operation;
• generating a mixed reality image in which a virtual object is superimposed on a captured image;
• and
• determining whether or not the user can visually recognize the mobile terminal based on whether or not the mobile terminal can be visually recognized in the mixed reality image without being blocked by the virtual object based on a positional relationship between the mobile terminal and the virtual object in the mixed reality image,
• wherein, in the deciding, a first operation mode, in which the application operation is performed by the user operating the mobile terminal, the operation mode is decided in a case where it is determined in the determining that the user can visually recognize the mobile terminal, and a second operation mode, in which a virtual user interface (UI) relating to an operation is displayed on the display device and the application operation is performed by the user operating the virtual UI, is decided as the operation mode in a case where it is determined in the determining that the user cannot visually recognize the mobile terminal.

A non-transitory computer readable medium that stores a program, wherein the program causes a computer to execute a control method of an information processing apparatus, and

• the control method comprises:
• generating an image to be displayed on a display device that is mountable to the head of a user;
• deciding an operation mode of an application;
• accepting an input operation from a mobile terminal that the user is capable of holding and operating, and performing an application operation on the basis of the operation mode and the input operation;
• generating a mixed reality image in which a virtual object is superimposed on a captured image;
• and
• determining whether or not the user can visually recognize the mobile terminal based on whether or not the mobile terminal can be visually recognized in the mixed reality image without being blocked by the virtual object based on a positional relationship between the mobile terminal and the virtual object in the mixed reality image,
• wherein, in the deciding, a first operation mode, in which the application operation is performed by the user operating the mobile terminal, the operation mode is decided in a case where it is determined in the determining that the user can visually recognize the mobile terminal, and a second operation mode, in which a virtual user interface (UI) relating to an operation is displayed on the display device and the application operation is performed by the user operating the virtual UI, is decided as the operation mode in a case where it is determined in the determining that the user cannot visually recognize the mobile terminal.