# US 12,118,684 B2 · Fitness system for simulating a virtual fitness partner and methods for use therewith > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,118,684 B2 - Document ID: US12118684B2 - Status: Granted - Kind: B2 - Application number: 18/358,174 - Filed: 2023-07-25 - Published: 2023-11-16 - Granted: 2024-10-15 - Earliest claimed: 2018-04-05 - Assignees: Mark Hogue; Bruce Stuckman - Applicants: Mark Wayne Hogue; Bruce Edward Stuckman - Inventors: Mark Wayne Hogue - Web page: https://patentgrep.com/patent/US12118684B2 ## CPC - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking} - G02B 27/0172: {characterised by optical features} - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/012: {Head tracking input arrangements} - G16H 20/30: relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising - G16H 40/63: for local operation - H04L 67/306: User profiles - G02B 2027/0141: {characterised by the informative content of the display} - G02B 2027/0178: {Eyeglass type (eyeglass details G02C)} - H04W 4/80: Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72) ## Abstract A fitness system operates during a fitness activity of a user to generate a display. The display simulates the performance of the fitness activity by the virtual fitness partner. In particular, the virtual fitness partner is superimposed within images viewed by the user to simulate performance of the fitness activity by the virtual fitness partner while the user is engaged in the activity. Differences in perspective are calculated based on a virtual position of the virtual fitness partner and a measured pace of the user. In addition, the virtual fitness partner can take actions such as generating critical or encouraging visual and/or auditory feedback in response to triggering events, for example, when the user falls behind the virtual fitness partner or takes the lead. ## Claims ### Claim 1 A fitness system, comprising: at least one sensor that generates sensor data; a motion data generator that processes the sensor data to generate motion data, wherein the motion data indicates a pace of a user during a fitness activity utilizing an article of fitness equipment; a fitness partner generator that generates display data associated with a virtual fitness partner for display on a display screen, wherein the display data is based on: appearance data corresponding to an appearance of the virtual fitness partner, wherein the appearance data is determined based on fitness partner profile data; perspective data calculated based on a virtual position vector, wherein the virtual position vector is determined based on calculating relative velocity data based on the motion data and a partner pace, and wherein the virtual position vector indicates a distance and a direction of a virtual position of the virtual fitness partner relative to a position of the user; and event trigger monitoring data based on the motion data, wherein the event trigger monitoring data indicates one of a plurality of fitness partner actions in response to the fitness partner generator determining that a corresponding one of a plurality of event trigger conditions has been met; wherein the display screen is configured to display the virtual fitness partner within a display region and in accordance with the display data to virtually superimpose the virtual fitness partner within images viewed by the user to simulate performance of the fitness activity by the virtual fitness partner along a virtual transit of the user, and wherein the virtual fitness partner is depicted to perform the one of the plurality of fitness partner actions in accordance with the event trigger monitoring data. ### Claim 2 (depends on 1) The fitness system of claim 1, wherein the article of fitness equipment includes a tread mill. ### Claim 3 (depends on 1) The fitness system of claim 1, wherein the article of fitness equipment includes an exercise bicycle. ### Claim 4 (depends on 1) The fitness system of claim 1, wherein the one of the plurality of event trigger conditions includes determining that a magnitude of a scalar projection of the virtual position vector onto a vector corresponding to the motion data exceeds a lag threshold. ### Claim 5 (depends on 4) The fitness system of claim 4, wherein the lag threshold is indicated in the fitness partner profile data. ### Claim 6 (depends on 5) The fitness system of claim 5, wherein the corresponding one of the plurality of fitness partner actions includes changing a magnitude of the partner pace from a first pace to a second pace that is slower than the first pace. ### Claim 7 (depends on 1) The fitness system of claim 1, wherein the one of the plurality of event trigger conditions includes determining that a magnitude of a scalar projection of the virtual position vector onto a vector that is orthogonal to the motion data exceeds a gap threshold, and wherein the corresponding one of the plurality of fitness partner actions includes setting a first updated direction to generate a first updated partner velocity vector, wherein the magnitude of a first cross product of the first updated partner velocity vector and the motion data is nonzero. ### Claim 8 (depends on 7) The fitness system of claim 7, wherein a second one of the plurality of event trigger conditions includes determining that the magnitude of the scalar projection of the virtual position vector onto the vector that is orthogonal to the motion data no longer exceeds the gap threshold, and wherein a second one of the plurality of fitness partner actions corresponding to the second one of the plurality of event trigger conditions includes setting a second updated direction to generate a second updated partner velocity vector, wherein the magnitude of a second cross product of the second updated partner velocity vector and the motion data is less than the magnitude of the first cross product. ### Claim 9 (depends on 1) The fitness system of claim 1, wherein the one of the plurality of event trigger conditions includes detecting a change in motion data from a first direction to a second direction, and wherein the corresponding one of the plurality of fitness partner actions includes setting an updated direction of the virtual fitness partner in accordance with the change in motion data. ### Claim 10 (depends on 1) The fitness system of claim 1, wherein the one of the plurality of event trigger conditions includes determining that a derivative of the pace exceeds a pace derivative threshold. ### Claim 11 (depends on 1) The fitness system of claim 1, wherein the fitness partner profile data includes a plurality of partner velocity vectors and a corresponding plurality of interval times, and wherein the virtual transit of the virtual fitness partner is displayed in accordance with the plurality of partner velocity vectors and a corresponding plurality of interval times. ### Claim 12 (depends on 11) The fitness system of claim 11, wherein the plurality of partner velocity vectors and the corresponding plurality of interval times are generated based on a plurality of previous motion datas and a corresponding plurality of previous timestamps generated by the motion data generator in a previous performance of the fitness activity. ### Claim 13 (depends on 1) The fitness system of claim 1, wherein the fitness partner profile data is generated based on fitness activity history data associated with the user. ### Claim 14 (depends on 1) The fitness system of claim 1, wherein the fitness partner profile data is generated based on selecting one of a plurality of past performances of the fitness activity by the user. ### Claim 15 (depends on 1) The fitness system of claim 1, wherein the fitness partner profile data includes second user appearance data, and wherein the virtual fitness partner is displayed to resemble a second user based on the second user appearance data. ### Claim 16 A method for execution by a fitness system that includes a memory, and at least one processor, wherein the method is executed in conjunction with instructions stored by the memory that, when executed by the at least one processor, cause the fitness system to: generate motion data corresponding to motion of a user of fitness equipment during performance of a fitness activity, wherein the motion data indicates a pace; generate relative velocity data based on the motion data and a partner velocity vector, wherein the partner velocity vector is determined based on fitness partner profile data stored in the memory; determine a virtual position vector based on the relative velocity data that indicates a distance and a direction of a virtual position of a virtual fitness partner relative to the user; generate perspective data calculated based on the virtual position vector; generate event trigger monitoring data based on the motion data, wherein the event trigger monitoring data indicates one of a plurality of fitness partner actions in response to determining that a corresponding one of a plurality of event trigger conditions has been met; generate display data based on the perspective data, the event trigger monitoring data, and appearance data corresponding to the virtual fitness partner; and display the virtual fitness partner within a display region of a display device and in accordance with the display data to virtually superimpose the virtual fitness partner upon images viewed by the user to simulate performance of the fitness activity by the virtual fitness partner along a virtual transit of the user, and wherein the virtual fitness partner is depicted to perform the one of the plurality of fitness partner actions in accordance with the event trigger monitoring data. ### Claim 17 (depends on 16) The method of claim 16, wherein the fitness partner profile data is generated based on fitness activity history data associated with the user. ### Claim 18 (depends on 16) The method of claim 16, wherein the fitness partner profile data is generated based on selecting one of a plurality of past performances of the fitness activity by the user. ### Claim 19 (depends on 16) The method of claim 16, wherein the fitness partner profile data includes second user appearance data, and wherein the virtual fitness partner is displayed to resemble a second user based on the second user appearance data. ### Claim 20 (depends on 16) The method of claim 16, wherein the one of the plurality of event trigger conditions includes determining that a magnitude of a scalar projection of the virtual position vector onto a vector corresponding to the motion data exceeds a lag threshold.