Information processing system, non-transitory computer-readable storage medium having stored therein information processing program, information processing apparatus, and information processing method

A non-transitory computer-readable storage medium having stored therein an information processing program executed by a computer of an information processing apparatus that outputs to a display device a combined image obtained by combining a real image obtained by an image capturing apparatus sequentially capturing a real space and a virtual image obtained by a virtual camera placed in a virtual space sequentially capturing a virtual object, the information processing program causing the computer to:

• in a first control state, control at least either one of an image capturing direction of the virtual camera and a position of the virtual object in accordance with an image capturing direction of the image capturing apparatus so that the image capturing direction of the virtual camera and a virtual object direction that is a direction from the virtual camera to the virtual object are linked together at a first linkage level;
• based on a user operation, change the first control state to a second control state;
• in the second control state, control at least any one of a position of the virtual camera, the image capturing direction of the virtual camera, and the position of the virtual object in accordance with a position or the image capturing direction of the image capturing apparatus so that the virtual object viewed in the combined image is fixed in a simulated manner at a position in the real space;
• and
• in the first control state, if the image capturing direction of the image capturing apparatus is directed more upward than a first threshold or directed more downward than a second threshold in the real space, control at least either one of the image capturing direction of the virtual camera and the position of the virtual object so that the image capturing direction of the virtual camera and the virtual object direction are linked together at a second linkage level lower than the first linkage level.

An information processing apparatus that outputs to a display device a combined image obtained by combining a real image obtained by an image capturing apparatus sequentially capturing a real space and a virtual image obtained by a virtual camera placed in a virtual space sequentially capturing a virtual object, the information processing apparatus, comprising:

• at least one processor and a memory coupled thereto, the processor being configured to control the information processing apparatus to at least:
• in a first control state, control at least either one of an image capturing direction of the virtual camera and a position of the virtual object in accordance with an image capturing direction of the image capturing apparatus so that the image capturing direction of the virtual camera and a virtual object direction that is a direction from the virtual camera to the virtual object are linked together at a first linkage level;
• based on a user operation, change the first control state to a second control state;
• in the second control state, control at least any one of a position of the virtual camera, the image capturing direction of the virtual camera, and the position of the virtual object in accordance with a position or the image capturing direction of the image capturing apparatus so that the virtual object viewed in the combined image is fixed in a simulated manner at a position in the real space;
• and
• in the first control state, if the image capturing direction of the image capturing apparatus is directed more upward than a first threshold or directed more downward than a second threshold in the real space, control at least either one of the image capturing direction of the virtual camera and the position of the virtual object so that the image capturing direction of the virtual camera and the virtual object direction are linked together at a second linkage level lower than the first linkage level.

An information processing method executed by an information processing system that outputs to a display device a combined image obtained by combining a real image obtained by an image capturing apparatus sequentially capturing a real space and a virtual image obtained by a virtual camera placed in a virtual space sequentially capturing a virtual object, the information processing method comprising:

• in a first control state, controlling at least either one of an image capturing direction of the virtual camera and a position of the virtual object in accordance with an image capturing direction of the image capturing apparatus so that the image capturing direction of the virtual camera and a virtual object direction that is a direction from the virtual camera to the virtual object are linked together at a first linkage level;
• based on a user operation, changing the first control state to a second control state;
• in the second control state, controlling at least any one of a position of the virtual camera, the image capturing direction of the virtual camera, and the position of the virtual object in accordance with a position or the image capturing direction of the image capturing apparatus so that the virtual object viewed in the combined image is fixed in a simulated manner at a position in the real space;
• and
• in the first control state, if the image capturing direction of the image capturing apparatus is directed more upward than a first threshold or directed more downward than a second threshold in the real space, controlling at least either one of the image capturing direction of the virtual camera and the position of the virtual object so that the image capturing direction of the virtual camera and the virtual object direction are linked together at a second linkage level lower than the first linkage level.

An information processing system that outputs to a display device a combined image obtained by combining a real image obtained by an image capturing apparatus sequentially capturing a real space and a virtual image obtained by a virtual camera placed in a virtual space sequentially capturing a virtual object, the information processing system, comprising:

• at least one processor and a memory coupled thereto, the processor being configured to control the information processing system to at least:
• control an image capturing direction of the virtual camera in accordance with an image capturing direction of the image capturing apparatus;
• control a position of the virtual object in a form of following the image capturing direction of the virtual camera;
• based on a user operation, control the position of the virtual object so that the position of the virtual object is fixed in a simulated manner at a position in the real space;
• in a state where the position of the virtual object is fixed, control a position of the virtual camera in accordance with a position of the image capturing apparatus;
• and
• if the image capturing direction of the image capturing apparatus is directed more upward than a first threshold or directed more downward than a second threshold in the real space, control the position of the virtual object so that a degree to which the virtual object follows the image capturing direction of the virtual camera decreases.

A non-transitory computer-readable storage medium having stored therein an information processing program executed by a computer of an information processing apparatus that outputs to a display device a combined image obtained by combining a real image obtained by an image capturing apparatus sequentially capturing a real space and a virtual image obtained by a virtual camera placed in a virtual space sequentially capturing a virtual object, the information processing program causing the computer to:

• control an image capturing direction of the virtual camera in accordance with an image capturing direction of the image capturing apparatus;
• control a position of the virtual object in a form of following the image capturing direction of the virtual camera;
• based on a user operation, control the position of the virtual object so that the position of the virtual object is fixed in a simulated manner at a position in the real space;
• in a state where the position of the virtual object is fixed, control a position of the virtual camera in accordance with a position of the image capturing apparatus;
• and
• if the image capturing direction of the image capturing apparatus is directed more upward than a first threshold or directed more downward than a second threshold in the real space, control the position of the virtual object so that a degree to which the virtual object follows the image capturing direction of the virtual camera decreases.

An information processing apparatus that outputs to a display device a combined image obtained by combining a real image obtained by an image capturing apparatus sequentially capturing a real space and a virtual image obtained by a virtual camera placed in a virtual space sequentially capturing a virtual object, the information processing apparatus, comprising:

• at least one processor and a memory coupled thereto, the processor being configured to control the information processing system to at least:
• control an image capturing direction of the virtual camera in accordance with an image capturing direction of the image capturing apparatus;
• control a position of the virtual object in a form of following the image capturing direction of the virtual camera;
• based on a user operation, control the position of the virtual object so that the position of the virtual object is fixed in a simulated manner at a position in the real space;
• in a state where the position of the virtual object is fixed, control a position of the virtual camera in accordance with a position of the image capturing apparatus;
• and
• if the image capturing direction of the image capturing apparatus is directed more upward than a first threshold or directed more downward than a second threshold in the real space, control the position of the virtual object so that a degree to which the virtual object follows the image capturing direction of the virtual camera decreases.

An information processing method executed by an information processing system that outputs to a display device a combined image obtained by combining a real image obtained by an image capturing apparatus sequentially capturing a real space and a virtual image obtained by a virtual camera placed in a virtual space sequentially capturing a virtual object, the information processing method comprising:

• controlling an image capturing direction of the virtual camera in accordance with an image capturing direction of the image capturing apparatus;
• controlling a position of the virtual object in a form of following the image capturing direction of the virtual camera;
• based on a user operation, controlling the position of the virtual object so that the position of the virtual object is fixed in a simulated manner at a position in the real space;
• in a state where the position of the virtual object is fixed, controlling a position of the virtual camera in accordance with a position of the image capturing apparatus;
• and
• if the image capturing direction of the image capturing apparatus is directed more upward than a first threshold or directed more downward than a second threshold in the real space, controlling the position of the virtual object so that a degree to which the virtual object follows the image capturing direction of the virtual camera decreases.