# US 2024/0005701 A1 · Methods and apparatus for team classification in sports analysis

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2024/0005701 A1
- Document ID: US20240005701A1
- Status: Published application
- Kind: A1
- Application number: 18/254,818
- Filed: 2021-09-23
- Published: 2024-01-04
- Earliest claimed: 2021-09-23
- Assignees: Intel Corporation
- Applicants: Intel Corporation
- Inventors: Chenning Liu; Qiang Li; Wenlong Li; Yikai Fang; Hang Zheng; Jiansheng Chen
- Web page: https://patentgrep.com/patent/US20240005701A1

## CPC

- G06V 40/23: {Recognition of whole body movements, e.g. for sport training}
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/82: using neural networks
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

An example apparatus includes processor circuitry to extract features from image data obtained from a plurality of cameras, the extraction of features performed using a plurality of sequential neural network layers; in response to each of the plurality of sequential neural network layer extracting the features, identify the extracted features in a torso region of the image data via a plurality of attention modules; estimate body landmarks from image data to localize an area; generate an upper heatmap mask based on a geometric center of the image data; calculate a loss function for the image data based on a cross-entropy loss, a pixel-wise loss, and a triplet loss determined from the extracted features and the generated heatmap mask; select lowest correlated classes based on calculated correlations between pairs of a plurality of classes; and calculate voting scores for groups associated with the lowest correlated classes.

## Claims

### Claim 1

An apparatus comprising:
at least one memory;
instructions in the apparatus; and
processor circuitry to execute the instructions to:
extract features from image data obtained from a plurality of cameras, the extraction of features performed using a plurality of sequential neural network layers;
in response to each of the plurality of sequential neural network layer extracting the features, identify the extracted features in a torso region of the image data via a plurality of attention modules in parallel with each of the plurality of sequential neural network layers;
estimate body landmarks from image data to localize an area;
generate an upper heatmap mask based on a geometric center of the image data;
calculate a loss function for the image data based on a cross-entropy loss, a pixel-wise loss, and a triplet loss determined from the extracted features and the generated heatmap mask;
select lowest correlated classes based on calculated correlations between pairs of a plurality of classes; and
calculate voting scores for groups associated with the lowest correlated classes based on the image data and the loss function.

### Claim 2 (depends on 1)

The apparatus of claim 1, wherein the plurality of sequential neural network layers includes at least four sequential neural network layers, and the plurality of attention modules includes at least three attention modules.

### Claim 3 (depends on 1)

The apparatus of claim 1, wherein to determine the extracted features in a channel dimension, the processor circuitry is to:
perform global maximum pooling on the extracted features;
compress the extracted features via convolution;
perform a leaky rectified linear unit function;
recover feature channels to match number of the image data; and
normalize the extracted features with sigmoid function.

### Claim 4 (depends on 3)

The apparatus of claim 3, wherein to determine the extracted features in a spatial dimension, the processor circuitry is to:
scale the extracted features to a fourth size of image data;
transpose the extracted features;
determine a self-correlation factor by multiplying the scaled extracted features by the transposed extracted features;
multiply the self-correlation factor by convolution of the extracted features; and
perform convolution to match channel number of the extracted features and channel number of image data.

### Claim 5 (depends on 4)

The apparatus of claim 4, wherein the processor circuitry is to identify the extracted features in the torso region by combining the extracted features in the channel dimension and the extracted features in the spatial dimension.

### Claim 6 (depends on 1)

The apparatus of claim 1, wherein the processor circuitry is to estimate the body landmarks and generate the upper heatmap mask in parallel to the plurality of sequential neural network layers.

### Claim 7 (depends on 1)

The apparatus of claim 1, wherein the area is an upper torso area.

### Claim 8 (depends on 1)

The apparatus of claim 1, wherein the processor circuitry is to calculate correlation coefficients for pairs of classes, wherein each class is a team classification.

### Claim 9 (depends on 1)

The apparatus of claim 1, wherein the processor circuitry is to associate a label with the image data based on the voting score.

### Claim 10 (depends on 9)

The apparatus of claim 9, wherein the label is to identify a presence of a uniformed personnel.

### Claim 11 (depends on 1)

The apparatus of claim 1, wherein the processor circuitry is to group a player in a ground plane and determine player correspondence from the plurality of cameras, the instructions to associate bounding boxes of a player from the image data of the plurality of cameras.

### Claim 12 (depends on 11)

The apparatus of claim 11, wherein the processor circuitry is to calculate the voting scores based on an area of a current bounding box, an area of a largest bounding box in the image data, width of the current bounding box, and height of the current bounding box.

### Claim 13 (depends on 12)

The apparatus of claim 12, wherein the processor circuitry is to associate a team label with the image data for the player in the current bounding box.

### Claim 14

A non-transitory computer readable storage medium comprising instructions that, when executed, cause at least one processor to:
extract features from image data obtained from a plurality of cameras, the extraction of features performed using a plurality of sequential neural network layers;
in response to each of the plurality of sequential neural network layer extracting the features, identify the extracted features in a torso region of the image data via a plurality of attention modules in parallel with each of the plurality of sequential neural network layers;
estimate body landmarks from image data to localize an area;
generate an upper heatmap mask based on a geometric center of the image data;
calculate a loss function for the image data based on a cross-entropy loss, a pixel-wise loss, and a triplet loss determined from the extracted features and the generated heatmap mask;
select lowest correlated classes based on calculated correlations between pairs of a plurality of classes; and
calculate voting scores for groups associated with the lowest correlated classes based on the image data and the loss function.

### Claim 15 (depends on 14)

The non-transitory computer readable storage medium of claim 14, wherein the plurality of sequential neural network layers includes at least four sequential neural network layers, and the plurality of attention modules includes at least three attention modules.

### Claim 16 (depends on 14)

The non-transitory computer readable storage medium of claim 14, wherein to determine the extracted features in a channel dimension, the instructions, when executed, cause the at least one processor to:
perform global maximum pooling on the extracted features;
compress the extracted features via convolution;
perform a leaky rectified linear unit function;
recover feature channels to match number of the image data; and
normalize the extracted features with sigmoid function.

### Claim 17 (depends on 16)

The non-transitory computer readable storage medium of claim 16, wherein to determine the extracted features in a spatial dimension, the instructions, when executed, cause the at least one processor to:
scale the extracted features to a fourth size of image data;
transpose the extracted features;
determine a self-correlation factor by multiplying the scaled extracted features by the transposed extracted features;
multiply the self-correlation factor by convolution of the extracted features; and
perform convolution to match channel number of the extracted features and channel number of image data.

### Claim at position 18

18-26. (canceled)

### Claim 27

A method comprising:
extracting features from image data obtained from a plurality of cameras, the extraction of features performed using a plurality of sequential neural network layers;
in response to each of the plurality of sequential neural network layer extracting the features, identifying the extracted features in a torso region of the image data via a plurality of attention modules in parallel with each of the plurality of sequential neural network layers;
estimating body landmarks from image data to localize an area;
generating an upper heatmap mask based on a geometric center of the image data;
calculating a loss function for the image data based on a cross-entropy loss, a pixel-wise loss, and a triplet loss determined from the extracted features and the generated heatmap mask;
selecting lowest correlated classes based on calculated correlations between pairs of a plurality of classes; and
calculating voting scores for groups associated with the lowest correlated classes based on the image data and the loss function.

### Claim at position 20

28-33. (canceled)

### Claim 34 (depends on 27)

The method of claim 27, further including calculating correlation coefficients for pairs of classes, wherein each class is a team classification.

### Claim 35 (depends on 27)

The method of claim 27, further including associating a label with the image data based on the voting score.

### Claim at position 23

36-66. (canceled)
