# US 11,850,425 B2 · Synchronizing output from a fitness apparatus with media content

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 11,850,425 B2
- Document ID: US11850425B2
- Status: Granted
- Kind: B2
- Application number: 16/585,717
- Filed: 2019-09-27
- Published: 2020-12-31
- Granted: 2023-12-26
- Earliest claimed: 2019-06-25
- Assignees: Katalyst Interactive, Inc.
- Applicants: Katalyst Interactive, Inc.
- Inventors: Björn Erich Woltermann
- Web page: https://patentgrep.com/patent/US11850425B2

## CPC

- A61N 1/36034: {specified by the stimulation parameters}
- A61N 1/0452: {Specially adapted for transcutaneous muscle stimulation [TMS]}
- A61N 1/18: Applying electric currents by contact electrodes
- A61N 1/36003: {of motor muscles, e.g. for walking assistance}
- A61N 1/36014: {External stimulators, e.g. with patch electrodes (external pacemakers A61N1/3625)}
- A61N 1/3603: {Control systems}
- A61N 1/37282: {characterised by communication with experts in remote locations using a network}
- A63B 24/0062: {Monitoring athletic performances, e.g. for determining the work of a user on an exercise apparatus, the completed jogging or cycling distance}
- A63B 24/0075: {Means for generating exercise programmes or schemes, e.g. computerized virtual trainer, e.g. using expert databases}
- A63B 71/0622: {Visual, audio or audio-visual systems for entertaining, instructing or motivating the user}
- G06F 16/9035: Filtering based on additional data, e.g. user or group profiles
- G11B 27/10: Indexing; Addressing; Timing or synchronising; Measuring tape travel
- G11B 27/102: {Programmed access in sequence to addressed parts of tracks of operating record carriers (access by moving the head G11B3/08, G11B5/54, G11B7/085, G11B21/022; by moving the record carrier G11B15/005, G11B17/005, by driving of both record carrier and head G11B15/1816)}
- G11B 27/34: Indicating arrangements {(indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D)}
- H04N 5/2222: {Prompting}
- H04N 5/77: between a recording apparatus and a television camera
- H04N 21/43079: {of additional data with content streams on multiple devices}
- H04N 21/4312: {involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations}
- H04N 21/4325: {by playing back content from the storage medium (reproduction of recorded television signals H04N5/76; reproduction of recorded television signals H04N9/79)}
- H04N 21/43615: {Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals (home Audio Video Interoperability [HAVI] data switching networks H04L12/2805)}
- H04N 21/439: Processing of audio elementary streams
- H04N 21/44: Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N 21/6587: Control parameters, e.g. trick play commands, viewpoint selection
- A61N 1/0484: {Garment electrodes worn by the patient}
- A63B 2024/0068: {Comparison to target or threshold, previous performance or not real time comparison to other individuals}
- A63B 2071/0625: {Emitting sound, noise or music}
- A63B 2071/065: {Visualisation of specific exercise parameters}
- A63B 2213/004: with electrotherapy
- A63B 2220/806: Video cameras
- A63B 2220/808: Microphones
- A63B 2225/20: with means for remote communication, e.g. internet or the like
- H04N 21/2187: Live feed
- H04N 21/8547: involving timestamps for synchronizing content
- A61N: ELECTROTHERAPY; MAGNETOTHERAPY; RADIATION THERAPY; ULTRASOUND THERAPY (measurement of bioelectric currents A61B; surgical instruments, devices or methods for transferring non-mechanical forms of energy to or from the body A61B18/00; anaesthetic apparatus in general A61M; incandescent lamps H01K; infrared radiators for heating H05B)
- A63B: APPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT (apparatus for passive exercising, massage A61H)
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Fitness apparatus output can be synchronized with media content that features an instructor conducting a workout session. An instructor conducts a workout session while using a recording device to generate media data, and while a computing device executes a client application to generate command data for controlling aspects of output provided by a fitness apparatus. The media data and the command data generated by the instructor can be transmitted to user computing devices over a network. Users may consume the media data on-demand, or as a live stream while the instructor is conducting the workout session. Operations specified in the command data are driven by the timestamps in the media data to synchronize the output provided by a user's fitness apparatus with the visible cues and/or audible cues of the instructor in the media content. The intensity level of the fitness apparatus output may also be customized to the user.

## Claims

### Claim 1

A device comprising:
a processor; and
memory storing computer-executable instructions that, when executed by the processor, cause the device to:
receive, over a network from a remote system, media data corresponding to media content featuring an instructor conducting a workout session;
receive, over the network from the remote system, command data associated with the media data, the command data specifying:
operations; and
timestamps within the media data at which the operations are to be performed during playback of the media data on the device;
begin the playback of the media data for output of the media content on an output device;
determine that the playback of the media data has reached a first timestamp of the timestamps specified in the command data; and
in response to determining that the playback of the media data has reached the first timestamp, perform a first operation, of the operations specified in the command data, associated with the first timestamp, wherein the first operation comprises sending a command to an electrical muscle stimulation (EMS) suit to start delivering electrical impulses via an array of electrodes of the EMS suit for durations of seconds followed by rest periods for a defined number of repetitions associated with a current exercise movement, wherein delivery of the electrical impulses and the rest periods occur in synchronization with the media content.

### Claim 2 (depends on 1)

The device of claim 1, wherein the electrical impulses are delivered at relative intensity levels based on per-channel settings of electrical impulse intensities for respective channels of an array of channels associated with the array of electrodes .

### Claim 3 (depends on 1)

The device of claim 1, further comprising a wireless communication interface, wherein the sending of the command to the EMS suit comprises wirelessly sending the command via the wireless communication interface to the EMS suit.

### Claim 4 (depends on 1)

The device of claim 1, wherein the output device comprises a display, and wherein the computer-executable instructions, when executed by the processor, further cause the device to, in response to determining that the playback of the media data has reached the first timestamp:
perform a second operation, of the operations specified in the command data, associated with the first timestamp, wherein the second operation comprises causing presentation of a name of the current exercise movement and a time remaining until a next exercise movement on the display.

### Claim 5 (depends on 1)

The device of claim 1, wherein:
the media data is received, over the network, as a live data stream while the instructor is conducting the workout session; and
the live data stream includes:
a video data stream corresponding to video content that represents, at least in part, the instructor demonstrating exercises; and
an audio data stream corresponding to audio content that represents, at least in part, the instructor issuing verbal directions for the exercises,
wherein the timestamps are included in at least one of the video data stream or the audio data stream.

### Claim 6 (depends on 1)

The device of claim 1, wherein:
the media data is received as a pre-recorded data stream, the pre-recorded data stream having been recorded at a time prior to the device receiving the media data over the network; and
the timestamps are encoded in the pre-recorded data stream.

### Claim 7 (depends on 1)

The device of claim 1, wherein:
the command data is received as at least one of a JavaScript Object Notation (JSON) file, an Extensible Markup Language (XML) file, a YAML Ain′t Markup Language (YAML) file, a comma-separated values (CSV) file, a plain text file, or an array data structure; and
the computer-executable instructions, when executed by the processor, further cause the device to identify the first operation associated with the first timestamp based at least in part on executing the at least one of the JSON file, the XML file, the YAML file, the CSV file, the plaintext file, or the array data structure during the playback of the media data.

### Claim 8

A method comprising:
receiving, by a device, over a network from a remote system, media data corresponding to media content featuring an instructor conducting a workout session;
receiving, by the device, over the network from the remote system, command data associated with the media data, the command data specifying:
operations; and
timestamps within the media data at which the operations are to be performed during playback of the media data on the device;
beginning the playback of the media data for output of the media content on an output device;
determining, by the device, that the playback of the media data has reached a first timestamp of the timestamps specified in the command data; and
performing, by the device, a first operation, of the operations specified in the command data, associated with the first timestamp, wherein the first operation comprises sending a command to an electrical muscle stimulation (EMS) suit to start delivering electrical impulses via an array of electrodes of the EMS suit for durations of seconds followed by rest periods for a defined number of repetitions associated with a current exercise movement, wherein delivery of the electrical impulses and the rest periods occur in synchronization with the media content.

### Claim 9 (depends on 8)

The method of claim 8, further comprising casting the media content to a peripheral display system during the playback of the media data, the peripheral display system being separate from the device.

### Claim 10 (depends on 8)

The method of claim 8, wherein the output device comprises a display, the method further comprising:
performing, by the device, a second operation, of the operations specified in the command data, associated with the first timestamp, wherein the second operation comprises causing presentation of a name of the current exercise movement and a time remaining until a next exercise movement on the display.

### Claim 11 (depends on 8)

The method of claim 8, wherein:
the media data is received, over the network, as a live data stream while the instructor is conducting the workout session; and
the live data stream includes:
a video data stream corresponding to video content that represents, at least in part, the instructor demonstrating exercises; and
an audio data stream corresponding to audio content that represents, at least in part, the instructor issuing verbal directions for the exercises,
wherein the timestamps are included in at least one of the video data stream or the audio data stream.

### Claim 12 (depends on 8)

The method of claim 8, wherein:
the media data is received as a pre-recorded data stream, the pre-recorded data stream having been recorded at a time prior to the device receiving the media data over the network; and
the timestamps are encoded in the pre-recorded data stream.

### Claim 13

One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by a processor of a device, cause the device to:
receive, over a network from a remote system, media data corresponding to media content featuring an instructor conducting a workout session;
receive, over the network from the remote system, command data associated with the media data, the command data specifying:
operations; and
timestamps within the media data at which the operations are to be performed during playback of the media data on the device;
begin the playback of the media data for output of the media content on an output device;
determine that the playback of the media data has reached a first timestamp of the timestamps specified in the command data; and
perform a first operation, of the operations specified in the command data, associated with the first timestamp, wherein the first operation comprises sending a command to an electrical muscle stimulation (EMS) suit to start delivering electrical impulses via an array of electrodes of the EMS suit for durations of seconds followed by rest periods for a defined number of repetitions associated with a current exercise movement, wherein delivery of the electrical impulses and the rest periods occur in synchronization with the media content.

### Claim 14 (depends on 13)

The one or more non-transitory computer-readable media of claim 13, wherein the electrical impulses are delivered at relative intensity levels based on per-channel settings of electrical impulse intensities for respective channels of an array of channels associated with the array of electrodes.

### Claim 15 (depends on 13)

The one or more non-transitory computer-readable media of claim 13, wherein the sending of the command to the EMS suit comprises wirelessly sending the command via a wireless communication interface of the device to the EMS suit.

### Claim 16 (depends on 13)

The one or more non-transitory computer-readable media of claim 13, wherein the output device comprises a display, and wherein the computer-executable instructions, when executed by the processor, further cause the device to:
perform a second operation, of the operations specified in the command data, associated with the first timestamp, wherein the second operation comprises causing presentation of a name of the current exercise movement and a time remaining until a next exercise movement on the display.

### Claim 17 (depends on 13)

The one or more non-transitory computer-readable media of claim 13, wherein the device is designated as a master device among a group of devices collocated in an environment, the group of devices including the device and at least a second device designated as a slave device, and wherein the computer-executable instructions, when executed by the processor, further cause the device to:
iteratively send timestamp data to the second device during the playback of the media data on the device, the timestamp data indicating a current timestamp reached during the playback of the media data on the device.

### Claim 18 (depends on 1)

The device of claim 1, wherein the sending of the command to the EMS suit causes the electrical impulses to be delivered via the array of electrodes in synchronization with a visual demonstration or a verbal utterance of the instructor exhibited in the media content.

### Claim 19 (depends on 1)

The device of claim 1, wherein the sending of the command occurs before a point in the media content when a muscle contraction should start to account for latency of sending commands to the EMS suit.

### Claim 20 (depends on 1)

The device of claim 1, wherein the electrical impulses are delivered in accordance with intensity values that are a fraction of, or a multiple of, corresponding intensity values associated with the instructor.