# US 12,770,282 B2 · Joint angle determination under limited visibility

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,770,282 B2
- Document ID: US12770282B2
- Status: Granted
- Kind: B2
- Application number: 18/257,251
- Filed: 2021-12-16
- Published: 2024-02-08
- Granted: 2026-10-20
- Earliest claimed: 2020-12-21
- Assignees: Healthcare Outcomes Performance Company Limited
- Applicants: Healthcare Outcomes Performance Company Limited
- Inventors: Huy Quoc Phan; Thomas Harte; Axel Sylvan
- Web page: https://patentgrep.com/patent/US12770282B2

## CPC

- G06T 7/60: Analysis of geometric attributes
- G06V 10/7715: {Feature extraction, e.g. by transforming the feature space, e.g. multi-dimensional scaling [MDS]; Mappings, e.g. subspace methods}
- G06V 10/82: using neural networks
- G06T 2207/20221: Image fusion; Image merging
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Provided herein are methods and systems for determining joint information, such as joint angle and range of movement, quickly and accurately without using cumbersome, specialized equipment even when visibility to the joint may be limited. Such methods and systems may be achieved by using computationally efficient approaches that are accurate and robust enough to handle noises and occlusions. The methods and systems provided herein may provide quick, low-cost, on-demand approaches to obtain information about the joint health of a patient. The joint angle information may facilitate diagnosis, treatment, prognosis, and/or rehabilitation of the patient from a joint injury, pain, or discomfort by providing quick, actionable information to the healthcare provider.

## Claims

### Claim 1

A computer-implemented method for determining an angle in an object of interest, the method comprising:
(a) obtaining an image or a video of the object of interest;
(b) generating a plurality of key point heatmaps and a plurality of segment heatmaps from the image or the video;
(c) blending at least one of the plurality of key point heatmaps with at least one of the plurality of segment heatmaps to generate at least one blended heatmap;
(d) extracting features from the at least one blended heatmap, wherein the extracted features comprise extracted segments, and wherein the extracted segments are extracted using a line detection method; and
(e) determining the angle in the object of interest by calculating an angle formed by the extracted features, wherein the angle is formed by the extracted segments.

### Claim 2 (depends on 1)

The method of claim 1, wherein the extracted features comprise key points extracted from the at least one blended heatmap, wherein the angle formed by the extracted key points is defined by at least two of the extracted segments, wherein the at least two of the extracted segments are formed by connecting the extracted key points.

### Claim 3 (depends on 1)

The method of claim 1, wherein the extracted segments are extracted using Hough transform.

### Claim 4 (depends on 1)

The method of claim 1, wherein the object of interest comprises a joint of a subject.

### Claim 5 (depends on 1)

The method of claim 1, wherein generating the plurality of key point heatmaps and the plurality of segment heatmaps in step (b) uses a deep neural network.

### Claim 6 (depends on 5)

The method of claim 5, wherein the deep neural network comprises a convolutional network.

### Claim 7 (depends on 1)

The method of claim 1, wherein the plurality of key point heatmaps represents landmarks on the image or the video of the object of interest.

### Claim 8 (depends on 1)

The method of claim 1, wherein the plurality of segment heatmaps represents segments along a body part adjacent to a joint.

### Claim 9 (depends on 7)

The method of claim 7, wherein one of the extracted segments connects at least two of the landmarks along a body part adjacent to a joint.

### Claim 10 (depends on 5)

The method of claim 5, wherein step (b) further comprises generating a combined negative heatmap from the image or the video for training the deep neural network.

### Claim 11 (depends on 1)

The method of claim 1, wherein the at least one of the plurality of key point heatmaps that represents a key point spatially adjacent to a segment represented by the at least one of the plurality of segment heatmaps.

### Claim 12 (depends on 1)

The method of claim 1, wherein the blending provides improved handling of a noisy heatmap or a missing heatmap.

### Claim 13 (depends on 2)

The method of claim 2, wherein extracting the key points uses at least one of non-maximum suppression, blob detection, or heatmap sampling.

### Claim 14 (depends on 2)

The method of claim 2, wherein extracting the key points comprises selecting coordinates with highest pixel intensity in the at least one blended heatmap.

### Claim 15 (depends on 2)

The method of claim 2, wherein at least three key points are extracted.

### Claim 16

A computer-based system for determining an angle in an object of interest, the system comprising:
(a) a processor;
(b) a non-transitory medium comprising a computer program configured to cause the processor to:
(i) obtain an image or a video of the object of interest and input the image or the video into a computer program;
(ii) generate, using the computer program, a plurality of key point heatmaps and a plurality of segment heatmaps from the image or the video;
(iii) blend, using the computer program, at least one of the plurality of key point heatmaps with at least one of the plurality of segment heatmaps to generate at least one blended heatmap;
(iv) extract, using the computer program, features from the at least one blended heatmap, wherein the extracted features comprise extracted segments, wherein the extracted segments are extracted using a line detection method; and
(v) determine, using the computer program, the angle in the object of interest by calculating an angle formed by the extracted features, wherein the angle is formed by the extracted segments.

### Claim 17 (depends on 16)

The system of claim 16, wherein the extracted features comprise key points extracted from the at least one blended heatmap, wherein the angle formed by the extracted key points is defined by at least two of the extracted segments, wherein the at least two of the extracted segments are formed by connecting the extracted key points.

### Claim 18 (depends on 7)

The method of claim 7, wherein the landmarks comprise a joint and at least one body part adjacent to the joint.

### Claim 19 (depends on 1)

The method of claim 1, further comprising generating an output comprising the angle in the object of interest.

### Claim 20 (depends on 1)

The method of claim 1, wherein the blending comprises taking an average of pixel intensity at each corresponding coordinate of the at least one of the plurality of key point heatmaps and the at least one of the plurality of segment heatmaps.