Three-dimensional user interface apparatus and three-dimensional operation method

A three-dimensional operation method executed by at least one computer, comprising:

• acquiring three-dimensional information from a three-dimensional sensor;
• calculating three-dimensional position information on a three-dimensional coordinate space regarding a specific part of a target person by using the acquired three-dimensional information;
• generating virtual three-dimensional object data indicating a virtual three-dimensional object disposed in the three-dimensional coordinate space;
• acquiring state information of the specific part of the target person;
• specifying a predetermined process to be performed from among a plurality of predetermined processes on the basis of a combination of the acquired state information and a change in the three-dimensional position information;
• performing the specified predetermined process on the virtual three-dimensional object data;
• and
• displaying a virtual three-dimensional object corresponding to the virtual three-dimensional object data on which the predetermined process has been performed, on a display unit,
• wherein specifying the predetermined process comprises determining whether the specific part of the target person is present within a predetermined three-dimensional range with reference to the virtual three-dimensional object on the basis of the calculated three-dimensional position information, and determining whether the predetermined process is to be performed according to a result of the determination,
• wherein specifying the predetermined process comprises detecting a movement of the specific part of the target person from within the predetermined three-dimensional range to the outside of the predetermined three-dimensional range, and specifying, as the predetermined process, a movement process or a rotation process corresponding to a distance and a direction between a position within the predetermined three-dimensional range and a position outside the predetermined three-dimensional range before and after the movement.

A non-transitory computer-readable storage medium storing a program for causing at least one computer to execute a three-dimensional operation method, the three-dimensional operation method comprising:

• acquiring three-dimensional information from a three-dimensional sensor;
• calculating three-dimensional position information on a three-dimensional coordinate space regarding a specific part of a target person by using the acquired three-dimensional information;
• generating virtual three-dimensional object data indicating a virtual three-dimensional object disposed in the three-dimensional coordinate space;
• acquiring state information of the specific part of the target person;
• specifying a predetermined process to be performed from among a plurality of predetermined processes on the basis of a combination of the acquired state information and a change in the three-dimensional position information;
• performing the specified predetermined process on the virtual three-dimensional object data;
• and
• displaying a virtual three-dimensional object corresponding to the virtual three-dimensional object data on which the predetermined process has been performed, on a display unit,
• wherein specifying the predetermined process comprises determining whether the specific part of the target person is present within a predetermined three-dimensional range with reference to the virtual three-dimensional object on the basis of the calculated three-dimensional position information, and determining whether the predetermined process is to be performed according to a result of the determination,
• wherein specifying the predetermined process comprises detecting a movement of the specific part of the target person from within the predetermined three-dimensional range to the outside of the predetermined three-dimensional range, and specifying, as the predetermined process, a movement process or a rotation process corresponding to a distance and a direction between a position within the predetermined three-dimensional range and a position outside the predetermined three-dimensional range before and after the movement.

A three-dimensional user interface apparatus comprising:

• a three-dimensional information acquisition unit that acquires three-dimensional information from a three-dimensional sensor;
• a position calculation unit that calculates three-dimensional position information on a three-dimensional coordinate space regarding a specific part of a target person by using the three-dimensional information acquired by the three-dimensional information acquisition unit;
• a virtual data generation unit that generates virtual three-dimensional object data indicating a virtual three-dimensional object disposed in the three-dimensional coordinate space;
• a state acquisition unit that acquires state information of the specific part of the target person;
• an operation specifying unit that specifies a predetermined process to be performed from among a plurality of predetermined processes on the basis of a combination of the state information acquired by the state acquisition unit and a change in the three-dimensional position information;
• an object processing unit that performs the predetermined process specified by the operation specifying unit on the virtual three-dimensional object data;
• and
• a display processing Unit that displays a virtual three-dimensional object corresponding to the virtual three-dimensional object data on which the predetermined process has been performed, on a display unit,
• wherein the operation specifying, unit measures a period in which state information acquired by the state acquisition unit and the three-dimensional position information are not changed, and in a case where the measured period exceeds a predetermined period, the operation specifying unit specifies, as the predetermined process, a process of adding display data of a function menu to data of a virtual three-dimensional space in which the virtual three-dimensional object is disposed.