# US 2026/0097309 A1 · Apparatus and method of video tracking

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0097309 A1
- Document ID: US20260097309A1
- Status: Published application
- Kind: A1
- Application number: 19/349,243
- Filed: 2025-10-03
- Published: 2026-04-09
- Earliest claimed: 2024-10-04
- Applicants: Sony Interactive Entertainment Inc.
- Inventors: Simon Andrew St. John Brislin; Nicholas Anthony Edward Ryan; Andrew Swann; Pritpal Singh Panesar; Andrew William Walker
- Web page: https://patentgrep.com/patent/US20260097309A1

## CPC

- A63F 13/54: involving acoustic signals, e.g. for simulating revolutions per minute [RPM] dependent engine sounds in a driving game or reverberation against a virtual wall
- H04N 21/44008: {involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream (arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)}
- A63F 13/497: Partially or entirely replaying previous game actions
- G06F 18/40: Software arrangements specially adapted for pattern recognition, e.g. user interfaces or toolboxes therefor
- H04N 21/23418: {involving operations for analysing video streams, e.g. detecting features or characteristics (television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59)}
- H04N 21/4394: {involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams (arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58)}
- A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A method of identifying a predetermined event that is audibly acknowledged within content comprises the steps of obtaining at least a first reference data item representing a predetermined event, detecting a pattern of peaks in audio of the content, the peaks being detected using a predefined process and predefined criteria, comparing the pattern of peaks with a respective reference pattern in the or each reference data item using a visual comparison process, and identifying that a predetermined event has occurred within the content if the pattern of peaks matches, to a predetermined matching threshold degree, a respective reference pattern of a data item representing the predetermined event.

## Claims

### Claim at position 1

-19. (canceled)

### Claim 20

A computer-implemented method, the method comprising:
obtaining reference audio data including audio acknowledgements associated with an occurrence of an event;
obtaining candidate audio acknowledgement data;
generating (i) a pattern of peaks from the candidate audio acknowledgement data, and (ii) one or more patterns of peaks from the reference audio data;
comparing a visual representation of the one or more patterns of peaks from the reference audio data; and
in response to comparing the visual representations of the pattern of peaks from the candidate audio acknowledgement data with the visual representation of the one or more patterns of peaks from the reference audio data, determining that the candidate audio acknowledgement data is associated with an occurrence of the event.

### Claim 21 (depends on 20)

The method of claim 20, wherein generating the pattern of peaks from the candidate audio acknowledgment data comprises:
generating a spectrogram of the candidate audio acknowledgement data; and
detecting peaks in the spectrogram of the candidate audio acknowledgement data as local maxima.

### Claim 22 (depends on 20)

The method of claim 20, wherein generating the one or more patterns of peaks from the reference audio data comprises:
storing a plurality of spectrograms corresponding to different types of events; and
detecting peaks in stored reference spectrograms.

### Claim 23 (depends on 20)

The method of claim 20, wherein comparing visual features of the pattern of peaks comprises:
evaluating structural similarity between binary images representing the pattern of peaks from the candidate audio acknowledgement data and binary images representing the one or more patterns of peaks from the reference spectrograms.

### Claim 24 (depends on 23)

The method of claim 23., wherein comparing the visual representation of the pattern of peaks comprises:
computing a similarity score based on correlation, Hamming distance, or both between the binary images.

### Claim 25 (depends on 24)

The method of claim 24, wherein determining that the candidate audio acknowledgement data is associated with the occurrence of the event comprises:
classifying the candidate audio acknowledgement data into one of a plurality of event types based on the comparison.

### Claim 26 (depends on 25)

The method of claim 25, wherein classifying the candidate audio acknowledgement into one of the plurality of event types comprises:
applying a threshold to the similarity score to confirm association with the event.

### Claim 27 (depends on 20)

The method of claim 20, further comprising:
normalizing the candidate audio acknowledgement data to reduce noise prior to generating the pattern of peaks.

### Claim 28 (depends on 20)

The method of claim 20, wherein the reference audio data comprises one or more audio acknowledgements associated with user interactions during gameplay, prerecorded reference sounds, or both.

### Claim 29 (depends on 20)

The method of claim 20, wherein the candidate audio acknowledgement data is captured from a microphone of a controller, headset, a mobile device, or a combination thereof.

### Claim 30 (depends on 20)

The method of claim 20, wherein the visual representation comprises a reduced-resolution spectrogram that includes local maxima as peaks.

### Claim 31 (depends on 20)

The method of claim 20, further comprising:
in response to determining that the candidate audio acknowledgement data is associated with an occurrence of the event, updating the reference audio data with new audio acknowledgements.

### Claim 32

A system comprising:
one or more computers and one or more storage devices on which are stored instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
obtaining reference audio data including audio acknowledgements associated with an occurrence of an event;
obtaining candidate audio acknowledgement data;
generating (i) a pattern of peaks from the candidate audio acknowledgement data, and (ii) one or more patterns of peaks from the reference audio data;
comparing a visual representation of the one or more patterns of peaks from the reference audio data; and
in response to comparing the visual representations of the pattern of peaks from the candidate audio acknowledgement data with the visual representation of the one or more patterns of peaks from the reference audio data, determining that the candidate audio acknowledgement data is associated with an occurrence of the event.

### Claim 33 (depends on 32)

The system of claim 32, wherein generating the pattern of peaks from the candidate audio acknowledgment data comprises:
generating a spectrogram of the candidate audio acknowledgement data; and
detecting peaks in the spectrogram of the candidate audio acknowledgement data as local maxima.

### Claim 34 (depends on 32)

The system of claim 32, wherein generating the one or more patterns of peaks from the reference audio data comprises:
storing a plurality of spectrograms corresponding to different types of events; and
detecting peaks in stored reference spectrograms.

### Claim 35 (depends on 32)

The system of claim 32, wherein comparing visual features of the pattern of peaks comprises:
evaluating structural similarity between binary images representing the pattern of peaks from the candidate audio acknowledgement data and binary images representing the one or more patterns of peaks from the reference spectrograms.

### Claim 36

One or more non-transitory computer storage media encoded with computer program instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:
obtaining reference audio data including audio acknowledgements associated with an occurrence of an event;
obtaining candidate audio acknowledgement data;
generating (i) a pattern of peaks from the candidate audio acknowledgement data, and (ii) one or more patterns of peaks from the reference audio data;
comparing a visual representation of the one or more patterns of peaks from the reference audio data; and
in response to comparing the visual representations of the pattern of peaks from the candidate audio acknowledgement data with the visual representation of the one or more patterns of peaks from the reference audio data, determining that the candidate audio acknowledgement data is associated with an occurrence of the event.

### Claim 37 (depends on 36)

The non-transitory media of claim 36, wherein generating the pattern of peaks from the candidate audio acknowledgment data comprises:
generating a spectrogram of the candidate audio acknowledgement data; and
detecting peaks in the spectrogram of the candidate audio acknowledgement data as local maxima.

### Claim 38 (depends on 36)

The non-transitory media of claim 36, wherein generating the one or more patterns of peaks from the reference audio data comprises:
storing a plurality of spectrograms corresponding to different types of events; and
detecting peaks in stored reference spectrograms.

### Claim 39 (depends on 36)

The non-transitory media of claim 36, wherein comparing visual features of the pattern of peaks comprises:
evaluating structural similarity between binary images representing the pattern of peaks from the candidate audio acknowledgement data and binary images representing the one or more patterns of peaks from the reference spectrograms.
