Motion control of a virtual environment

#Natural user-input (NUI) technologies aim to provide intuitive modes of interaction between computing systems and human beings. For example, a human subject's motion input may be recognized as a discrete gesture. The discrete gesture may be translated into a command to control a computing system. For example, the command may adjust a property of a virtual environment, such as in a video game or other virtual simulation. In some cases, control based on a discrete gesture may lack granularity and nuance, because a corresponding command may not change based on variations in motion when performing the discrete gesture. For example, a control command may not change based on whether the human subject performs a discrete gesture quickly or slowly. Furthermore, motions that do not narrowly conform to specific gestures may not produce the results a user expects in a virtual environment.

#This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to implementations that solve any or all disadvantages noted in any part of this disclosure.

#Various embodiments relating to controlling a virtual environment based on motion of a human subject are provided. In one embodiment, an optical flow of depth video of a depth camera imaging a human subject may be recognized. An energy field created by motion of the human subject may be generated as a function of the optical flow and specified rules of a physical simulation of a virtual environment. The energy field may be mapped to a virtual position in the virtual environment. A property of a virtual object in the virtual environment may be adjusted based on a plurality of energy elements of the energy field in response to the virtual object interacting with the virtual position of the energy field.

#FIG. 1 shows an environment in which NUI is used to control a computing or game system, in accordance with an embodiment of this disclosure.

#FIG. 2 shows an NUI pipeline in accordance with an embodiment of this disclosure.

#FIG. 3 shows a method for controlling a virtual environment in accordance with an embodiment of this disclosure.

#This description relates to controlling properties of a virtual environment, such as in a video game or other virtual simulation, based on motion of a human subject. More particularly, this description relates to recognizing motion of a human subject to augment intermediate mechanisms for controlling properties of a virtual environment. For example, an optical flow may be recognized from video (e.g., depth/IR/RGB) received from an NUI interface system imaging the human subject. In particular, the optical flow may be derived from a plurality of cells mapped to the video. Each cell may include a motion value indicating a magnitude, direction, and/or another property of motion in that cell recognized between consecutive image frames of the video. The optical flow may holistically define movement across the entire image frame, in contrast to a discrete gesture that may be performed by a particular body part of the human subject. In other words, the optical flow may be used to gauge the overall character of the motion of the human subject, whether or not the human subject is properly performing a particular gesture that the system is trained to recognize.

#The optical flow may be used to generate a dynamic energy field that is mapped to a virtual environment to control one or more properties of the virtual environment. Furthermore, the energy field may influence properties of virtual objects in the virtual environment that interact with the energy field. Motion of the human subject recognized in the form of the optical flow provides a robust input that may be used to augment intermediate mechanisms, such as a gesture or an energy field, in order to enhance control of the virtual environment. Such control may have an increased level of responsiveness, granularity, and/or nuance when compared to using discrete gestures alone.

#FIG. 1 shows aspects of an example environment 100. The illustrated environment is a living room or family room of a personal residence. However, the approaches described herein are equally applicable in other environments, such as retail stores and kiosks, restaurants, information kiosks, public-service environments, etc. In the environment 100, a home-entertainment system 102 is installed. The home-entertainment system includes a large-format display 104 and an NUI interface system 106, both operatively coupled to a computing system 108. The computing and NUI interface systems may be coupled via a wired link, a wireless link, or in another suitable manner. In the illustrated embodiment, the display presents computer-generated imagery (still images, video, graphical user interface elements, etc.). The computing system may be a video-game system; a multimedia system configured to play music and/or video; a general-purpose computing system used for internet browsing and productivity applications; and/or virtually any other type of computing system, including mobile computing systems, without departing from the scope of this disclosure.