# US 11,992,746 B1 · Hybrid display system for a force measurement assembly, an exercise device, or an interactive game > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,992,746 B1 - Document ID: US11992746B1 - Status: Granted - Kind: B1 - Application number: 18/138,112 - Filed: 2023-04-23 - Granted: 2024-05-28 - Earliest claimed: 2015-08-22 - Assignees: Bertec Corporation - Applicants: Bertec Corporation - Inventors: Necip Berme - Web page: https://patentgrep.com/patent/US11992746B1 ## CPC - A63B 71/0622: {Visual, audio or audio-visual systems for entertaining, instructing or motivating the user} - A63B 22/0012: {the exercises for arms and legs being functionally independent} - A63B 22/0076: {Rowing machines for conditioning the cardio-vascular system (seats moving during exercise A63B22/0087; training appliances for rowing or sculling on boats A63B69/06)} - A63B 22/02: with movable endless bands {, e.g. treadmills (other training appliances for running on the spot A63B69/0028)} - A63B 22/0292: {separate for each leg, e.g. dual deck} - A63B 22/0605: {performing a circular movement, e.g. ergometers} - A63B 22/0664: {performing an elliptic movement} - A63B 22/20: using rollers, wheels, castors or the like, {e.g. gliding means,} to be moved over the floor or other surface, {e.g. guide tracks,} during exercising - A63B 24/0075: {Means for generating exercise programmes or schemes, e.g. computerized virtual trainer, e.g. using expert databases} - A63B 24/0087: {Electric or electronic controls for exercising apparatus of groups A63B21/00 - A63B23/00, e.g. controlling load} - A63B 26/003: {for improving balance or equilibrium (balance beams A63B4/00; apparatus for rope dancing A63B7/08; balancing drums, balancing boards or the like A63B22/16)} - A63B 69/36: for golf - A63F 13/49: Saving the game status; Pausing or ending the game - A63B 2024/009: {the load of the exercise apparatus being controlled in synchronism with visualising systems, e.g. hill slope} - A63B 2024/0096: {using performance related parameters for controlling electronic or video games or avatars} - A63B 2071/0638: {Displaying moving images of recorded environment, e.g. virtual environment} - A63B 2071/0655: {Tactile feedback} - A63B 2071/0658: {Position or arrangement of display} - A63B 2071/0666: {worn on the head or face, e.g. combined with goggles or glasses} - A63B 2071/0677: {Input by image recognition, e.g. video signals} - A63B 2071/0683: {Input by handheld remote control} - A63B 2220/05: Image processing for measuring physical parameters - A63B 2220/13: Relative positions - A63B 2220/40: Acceleration - A63B 2220/51: Force - A63B 2220/801: Contact switches - A63B 2220/803: Motion sensors - A63B 2220/806: Video cameras - A63B 2225/20: with means for remote communication, e.g. internet or the like - A63B 2225/50: Wireless data transmission, e.g. by radio transmitters or telemetry - A63B 2230/60: muscle strain, i.e. measured on the user - A63B 2244/102: Boxing - A63F 2300/8082: Virtual reality - A63B: APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT (apparatus for passive exercising, massage A61H) - A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR ## Abstract A hybrid display system for a force measurement assembly, an exercise device, or an interactive game is disclosed herein. The hybrid display system includes a head-mounted visual display device having an output screen; at least one camera or a substantially transparent lens for enabling a system user to see one or more images of a real-world environment outside of the head-mounted visual display device; and a data processing device configured to generate a virtual reality environment and display the virtual reality environment on the output screen of the head-mounted visual display device, and the one or more images of the real-world environment being visible through the substantially transparent lens or the data processing device further configured to project at least a portion of the one or more images of the real-world environment captured by the at least one camera into the display of the virtual reality environment. ## Claims ### Claim 1 A hybrid display system for a force measurement assembly, an exercise device, or an interactive game, the hybrid display system comprising: a head-mounted visual display device having an output screen, the head-mounted visual display device configured to display one or more images on the output screen so that the one or more images are viewable by a system user; at least one camera or a substantially transparent lens for enabling the system user to see one or more images of a real-world environment outside of the head-mounted visual display device, the at least one camera configured to capture the one or more images of the real-world environment outside of the head-mounted visual display device, or the substantially transparent lens being provided on the head-mounted visual display device so that the system user is able to see the one or more images of the real-world environment outside of the head-mounted visual display device through the substantially transparent lens; and a data processing device operatively coupled to the head-mounted visual display device, the data processing device configured to generate a virtual reality environment and display the virtual reality environment on the output screen of the head-mounted visual display device, and the one or more images of the real-world environment being visible through the substantially transparent lens or the data processing device further configured to project at least a portion of the one or more images of the real-world environment captured by the at least one camera into the display of the virtual reality environment so that the system user is able to have a frame of reference in the real-world environment while he or she is immersed in the virtual reality environment; a sensor device, the sensor device being operatively coupled to the data processing device; wherein the system user is: (i) disposed on an exercise device that relates to the virtual reality environment on the output screen of the head-mounted visual display device, or (ii) disposed on a force measurement assembly, the exercise device or the force measurement assembly comprising the sensor device; wherein the data processing device is further configured to remove the display of the virtual reality environment from the output screen of the head-mounted visual display device in response to: (i) the system user no longer interacting with the exercise device, or (ii) the system user no longer interacting with the force measurement assembly; and wherein the data processing device is further configured to determine that the system user is no longer interacting with the exercise device or the force measurement assembly based upon a signal received from the sensor device. ### Claim 2 (depends on 1) The hybrid display system according to claim 1, wherein the system user is disposed on the exercise device that relates to the virtual reality environment on the output screen of the head-mounted visual display device, and the data processing device is further configured to remove the display of the virtual reality environment from the output screen of the head-mounted visual display device in response to the system user no longer interacting with the exercise device so that the system user is able to safely interact with the real-world environment after completing an exercise session. ### Claim 3 (depends on 2) The hybrid display system according to claim 2, wherein the exercise device on which the system user is disposed is selected from the group consisting of: (i) an exercise bike, (ii) a treadmill, (iii) a rowing machine, (iv) an elliptical machine, and (v) another type of exercise device. ### Claim 4 (depends on 1) The hybrid display system according to claim 1, wherein the system user is disposed on the force measurement assembly, and the data processing device is further configured to remove the display of the virtual reality environment from the output screen of the head-mounted visual display device in response to the system user no longer interacting with the force measurement assembly so that the system user is able to safely interact with the real-world environment after completing a testing and/or training session on the force measurement assembly; wherein the force measurement assembly on which the system user is disposed is selected from the group consisting of: (i) a force plate, (ii) a balance plate, (iii) an instrumented treadmill, and (iv) another type of force measurement assembly that is configured to measure one or more forces and/or moments exerted thereon by the system user. ### Claim 5 (depends on 4) The hybrid display system according to claim 4, wherein the force measurement assembly comprises the sensor device, and the sensor device is in a form of at least one force transducer; and wherein the data processing device is further configured to determine that the system user is no longer interacting with the force measurement assembly based upon a signal received from the at least one force transducer of the force measurement assembly. ### Claim 6 (depends on 1) The hybrid display system according to claim 1, wherein the hybrid display system is provided with the at least one camera configured to capture the one or more images of the real-world environment outside of the head-mounted visual display device, and the at least one camera is further configured to capture one or more body gestures from the system user indicating that the system user is no longer interacting with the exercise device or the force measurement assembly; and wherein the data processing device is further configured to determine that the system user is no longer interacting with the exercise device or the force measurement assembly based upon the one or more body gestures from the system user captured by the at least one camera. ### Claim 7 (depends on 1) The hybrid display system according to claim 1, wherein the system user is playing an interactive game, and the data processing device is further configured to remove the display of the virtual reality environment from the output screen of the head-mounted visual display device in response to the system user ceasing the playing of the interactive game. ### Claim 8 (depends on 1) The hybrid display system according to claim 1, wherein the one or more images of the real-world environment are intermixed with the display of the virtual reality environment on the output screen of the head-mounted visual display device. ### Claim 9 (depends on 1) The hybrid display system according to claim 1, wherein the virtual reality environment on the output screen of the head-mounted visual display device is located in a first section of the output screen of the head-mounted visual display device, the one or more images of the real-world environment are located in a second section of the output screen of the head-mounted visual display device, and the first section of the output screen of the head-mounted visual display device is separate from the second section of the output screen of the head-mounted visual display device. ### Claim 10 (depends on 1) The hybrid display system according to claim 1, wherein the hybrid display system is provided with the at least one camera configured to capture the one or more images of the real-world environment outside of the head-mounted visual display device, and the at least one camera is disposed on the head-mounted visual display device. ### Claim 11 (depends on 1) The hybrid display system according to claim 1, wherein the hybrid display system is provided with the at least one camera configured to capture the one or more images of the real-world environment outside of the head-mounted visual display device, and the at least one camera is a part of a motion capture system that is separate from the head-mounted visual display device. ### Claim 12 (depends on 1) The hybrid display system according to claim 1, wherein the head-mounted visual display device is in the form of an augmented reality headset, a virtual reality headset, or a mixed reality headset. ### Claim 13 (depends on 1) The hybrid display system according to claim 1, wherein the hybrid display system is provided with the at least one camera configured to capture the one or more images of the real-world environment outside of the head-mounted visual display device, the one or more images of the real-world environment captured by the at least one camera include one or more images of the system user performing one or more movements associated with an activity, and the data processing device is further configured to project the one or more images of the system user performing the one or more movements associated with the activity into the display of the virtual reality environment so that the system user is able to improve his or her performance by self-training. ### Claim 14 (depends on 13) The hybrid display system according to claim 13, wherein the one or more images of the system user performing the one or more movements associated with the activity projected into the display of the virtual reality environment by the data processing device comprise the system user represented in the form of an avatar. ### Claim 15 (depends on 13) The hybrid display system according to claim 13, wherein the one or more images of the system user performing the one or more movements associated with the activity projected into the display of the virtual reality environment by the data processing device comprise multiple attempts of the system user performing the one or more movements associated with the activity in order for the system user to be able to assess his or her repeatability of the one or more movements; and wherein the multiple attempts of the system user performing the one or more movements associated with the activity are superimposed on one another when displayed on the output screen of the head-mounted visual display device by the data processing device. ### Claim 16 (depends on 13) The hybrid display system according to claim 13, wherein the system user is disposed on the force measurement assembly, and the force measurement assembly includes: a top surface for receiving the body of the system user; and at least one force transducer, the at least one force transducer configured to sense one or more measured quantities and output one or more signals that are representative of forces and/or moments being applied to the top surface of the force measurement assembly by the system user. ### Claim 17 (depends on 16) The hybrid display system according to claim 16, wherein the data processing device is further configured to receive the one or more signals that are representative of the forces and/or moments being applied to the top surface of the force measurement assembly by the system user, and to convert the one or more signals into output forces and/or moments; and wherein the data processing device is further configured to generate one or more graphical representations of the output forces and/or moments determined from the force measurement assembly for superimposition onto the one or more images of the system user performing one or more movements associated with the activity and/or onto an object being manipulated by the system user while performing one or more movements associated with the activity, and the data processing device is additionally configured to display the superimposed output forces and/or moments on the head-mounted visual display device so that the system user is able to visualize the superimposed output forces and/or moments when wearing the head-mounted visual display device, and adjust his or her movements based upon feedback from the superimposed output forces and/or moments. ### Claim 18 (depends on 1) The hybrid display system according to claim 1, wherein the data processing device is further configured to determine limb movement output data for one or more limbs of the system user using the at least one camera, a motion capture system, and/or an inertial measurement unit disposed on the one or more limbs of the system user, and the data processing device is additionally configured to control one or more elements in the virtual reality environment on the output screen of the head-mounted visual display device using the limb movement output data determined from the at least one camera, the motion capture system, and/or the inertial measurement unit. ### Claim 19 A hybrid display system for a force measurement assembly, an exercise device, or an interactive game, the hybrid display system comprising: a head-mounted visual display device having an output screen, the head-mounted visual display device configured to display one or more images on the output screen so that the one or more images are viewable by a system user; at least one camera or a substantially transparent lens for enabling the system user to see one or more images of a real-world environment outside of the head-mounted visual display device, the at least one camera configured to capture the one or more images of the real-world environment outside of the head-mounted visual display device, or the substantially transparent lens being provided on the head-mounted visual display device so that the system user is able to see the one or more images of the real-world environment outside of the head-mounted visual display device through the substantially transparent lens; and a data processing device operatively coupled to the head-mounted visual display device, the data processing device configured to generate a virtual reality environment and display the virtual reality environment on the output screen of the head-mounted visual display device, and the one or more images of the real-world environment being visible through the substantially transparent lens or the data processing device further configured to project at least a portion of the one or more images of the real-world environment captured by the at least one camera into the display of the virtual reality environment so that the system user is able to have a frame of reference in the real-world environment while he or she is immersed in the virtual reality environment; wherein the system user is playing an interactive game, or is performing a training and/or testing session while wearing the head-mounted visual display device; wherein the data processing device is further configured to remove the display of the virtual reality environment from the output screen of the head-mounted visual display device in response to the system user ceasing the playing of the interactive game, or ceasing the performance of the training and/or testing session; and wherein the data processing device is further configured to determine that the system user has ceased playing the interactive game or has ceased performing the training and/or testing session based upon a signal received from the at least one camera configured to capture the one or more images of the real-world environment, and/or a signal received from at least another camera facing the system user that is separate from the head-mounted visual display device. ### Claim 20 (depends on 19) The hybrid display system according to claim 19, wherein the hybrid display system is provided with the at least one camera configured to capture the one or more images of the real-world environment outside of the head-mounted visual display device, and the at least one camera is further configured to capture one or more body gestures from the system user indicating that the system user has ceased playing the interactive game or has ceased performing the training and/or testing session; and wherein the data processing device is further configured to determine that the system user has ceased playing the interactive game or has ceased performing the training and/or testing session based upon the one or more body gestures from the system user captured by the at least one camera, the data processing device being programmed with image recognition software in order to recognize the one or more body gestures from the system user, and to determine that the one or more body gestures indicate that the system user has ceased playing the interactive game or has ceased performing the training and/or testing session. ### Claim 21 (depends on 1) The hybrid display system according to claim 1, further comprising an eye movement tracking device configured to control a display of the one or more images on the output screen. ### Claim 22 (depends on 2) The hybrid display system according to claim 2, wherein the exercise device comprises the sensor device, and the sensor device is in a form of a load sensor or a pedal sensor; and wherein the data processing device is further configured to determine that the system user is no longer interacting with the exercise device based upon a signal received from the load sensor or the pedal sensor. ### Claim 23 (depends on 5) The hybrid display system according to claim 5, wherein the data processing device is further configured to: determine a vertical force based upon the signal received from the at least one force transducer of the force measurement assembly; and determine that the system user is no longer interacting with the force measurement assembly when the vertical force determined from the signal of the at least one force transducer of the force measurement assembly is less than a body weight of the system user.