Trajectory detection and feedback system for tennis

A non-transitory computer readable medium configured to store a computer program used by a computer, the computer readable medium comprising:

• computer code configured to receive image data from one or more sensors worn on a user's head wherein the image data includes images of a real tennis ball being hit towards the user on a real tennis court;
• based upon the image data, computer code configured to determine a position of the real tennis ball as a first function of time;
• computer code configured to control generation of images of a virtual tennis ball as a second function of time which are output to a display worn on the user's head which projects images into at least one of a user's eyes to mimic a position of the real tennis ball as the function of time approaching the user wherein the images of the virtual tennis ball approaching the user are generated based upon the image data received by the one or more sensors;
• computer code configured to determine (1) speed, position, and orientation of a hand-held device as the second function of time based upon sensor data from one or more sensors, (2) when the virtual tennis ball is hit by the user based upon the position and the orientation of the hand-held device as the second function of time and a virtual position of the tennis ball as the second function of time, and (3) an orientation of the hand-held device relative to the virtual tennis ball when the virtual tennis ball is determined to be hit by the user;
• computer code configured to determine a spin of the virtual tennis ball based the speed of the hand-held device as the second function of time and the orientation of the hand-held device relative to the virtual tennis ball when the virtual tennis ball is determined to be hit by the user;
• computer code configured to calculate a trajectory for the virtual tennis ball based upon the speed of the hand-held device as the second function of time, the spin of the virtual tennis ball, and the orientation of the hand-held device relative to the virtual tennis ball when the virtual tennis ball is determined to be hit by the user;
• when the virtual tennis ball is determined to be hit by the user, computer code configured to trigger a haptic sensation generator in the hand-held device wherein the haptic sensation generator transmits one or more haptic sensations to the user which mimic a feel of the real tennis ball being hit;
• and
• computer code configured to control, based upon the trajectory, generation of second images of the virtual tennis ball which are output to the display worn on the user's head and which show the virtual position of the virtual tennis ball as the second function of time after the virtual tennis ball is determined to be hit moving away from the user according to the trajectory.

A system, comprising:

• one or more sensors configured to sense movement of a hand-held device and to provide sensor data indicative of the sensed movement;
• a display worn on a head of a user;
• a haptic sensor generator coupled to the hand-held device;
• and
• a logic device configured to control the display to deliver images of a virtual tennis ball into at least one eye of the user, the logic device configured to determine a speed and a position of the hand-held device based upon the sensor data as the hand-held device is swung by the user, the logic device configured to determine when the virtual tennis ball is hit by the user based upon the determined position and to determine an orientation of the hand-held device based upon the sensor data when the virtual tennis ball is determined to be hit by the user, wherein the logic device is configured to determine a spin of the virtual tennis ball based upon the speed of the hand-held device and the orientation of the hand-held device when the virtual tennis ball is determined to be hit by the user, wherein the logic device is configured to a calculate a trajectory for the virtual tennis ball based upon the speed of the hand-held device, the spin of the virtual tennis ball, and the orientation of the hand-held device relative to the virtual tennis ball when the virtual tennis ball is determined to be hit by the user, wherein the logic device is configured to trigger the haptic sensation generator to transmit one or more haptic sensations to the user which mimic a feel of a real tennis ball being hit when the virtual tennis ball is determined to be hit by the user, wherein the logic device is further configured to control the display based upon the calculated trajectory to deliver images of the virtual tennis ball into at least one eye of the user such that the virtual tennis ball appears to be moving away from the user according to the calculated trajectory.

A method, comprising:

• sensing movement of a hand-held device by one or more sensors thereby providing sensor data indicative of the sensed movement;
• controlling a display worn on a head of user to deliver images of a virtual tennis ball into at least one eye of the user;
• determining with a logic device a speed and a position of the hand-held device based upon the sensor data as the hand-held device is swung by the user;
• determining with the logic device when the virtual tennis ball is hit by the user based upon the determined position of the hand-held device;
• determining with the logic device an orientation of the hand-held device based upon the sensor data when the virtual tennis ball is determined to be hit by the user;
• determining with the logic device a spin of the virtual tennis ball based upon the speed of the hand-held device and the orientation of the hand-held device when the virtual tennis ball is determined to be hit by the user;
• calculating with the logic device a trajectory for the virtual tennis ball based upon the speed of the hand-held device, the spin of the virtual tennis ball, and the orientation of the hand-held device when the virtual tennis ball is determined to be hit by the user;
• triggering with the logic device a haptic sensation generator to transmit one or more haptic sensations to the user which mimic a feel of a real tennis ball being hit when the virtual tennis ball is determined to be hit by the user;
• and
• controlling the display based upon the calculated trajectory to deliver images of the virtual tennis ball into at least one eye of the user such that the virtual tennis ball appears to be moving away from the user according to the calculated trajectory.