# US 2026/0087646 A1 · Object identifications in images or videos

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0087646 A1
- Document ID: US20260087646A1
- Status: Published application
- Kind: A1
- Application number: 19/404,315
- Filed: 2025-12-01
- Published: 2026-03-26
- Earliest claimed: 2021-01-04
- Assignees: Hinge Health, Inc.
- Applicants: Hinge Health, Inc.
- Inventors: Louis Harbour; Bahareh Bafandeh Mayvan; Colin Joseph Brown; Jeffrey Rainy
- Web page: https://patentgrep.com/patent/US20260087646A1

## CPC

- G06T 7/11: Region-based segmentation
- G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments
- G06N 3/02: Neural networks
- G06T 7/12: Edge-based segmentation
- G06T 7/194: involving foreground-background segmentation
- G06V 10/26: Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06V 10/273: {removing elements interfering with the pattern to be recognised}
- G06V 10/82: using neural networks
- G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V 40/23: {Recognition of whole body movements, e.g. for sport training}
- G06N 20/00: Machine learning
- G06T 2207/10016: Video; Image sequence
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/30196: Human being; Person
- G06T 2207/30221: Sports video; Sports image
- G16H 20/30: relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G16H: HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA

## Abstract

An apparatus is provided. The apparatus includes a communications interface to receive raw data from an external source. The raw data includes a representation of a first object and a second object. The apparatus further includes a memory storage unit to store the raw data. In addition, the apparatus includes a neural network engine to receive the raw data. The neural network engine is to generate a segmentation map and a boundary map. The apparatus also includes a post-processing engine to identify the first object and the second object based on the segmentation map and the boundary map.

## Claims

### Claim 1

A method comprising:
generating, based on an analysis of an image that includes a first human and a second human,
(i) a segmentation map that indicates whether each pixel in the image is part of the first human or the second human, and
(ii) a boundary map that divides the first human from the second human;
identifying, based on an analysis of the segmentation map and the boundary map, fragments of the first human and the second human that arise from occlusions between the first human and the second human; and
selecting a first group of the fragments for the first human and a second group of the fragments for the second human.

### Claim 2 (depends on 1)

The method of claim 1, wherein said selecting comprises:
using multiple bone heatmaps, each of which corresponds to a different bone type, to identify connections between the fragments,
using multiple joint heatmaps, each of which corresponds to a different joint type, to assign a joint type to each of the fragments, and
clustering, based on the connections and the assigned joint types, the fragments into the first and second groups, such that the first human and the second human are associated with no more than a single joint of each type.

### Claim 3 (depends on 2)

The method of claim 2, further comprising:
characterizing, based on an analysis of the segmentation map and the multiple joint heatmaps, overlap of the first and second humans to produce information that is usable to determine whether the first human is overlapping the second human or the second human is overlapping the first human.

### Claim 4 (depends on 3)

The method of claim 3, wherein the information includes a number of joints present in an overlapping region and/or a type of joints present in the overlapping region.

### Claim 5 (depends on 3)

The method of claim 3,
wherein the method further comprises:
identifying, based on an analysis of the segmentation map, a region of interest that corresponds to an overlapping region; and
wherein the overlap is characterized only in the region of interest, so as to reduce computational resources needed to produce the information.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
generating a first skeletal representation of the first human based on the first group of the fragments and a second skeletal representation of the second human based on the second group of the fragments.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
searching for missing fragments by comparing the first and second groups of the fragments to a map of main fragments.

### Claim 8 (depends on 7)

The method of claim 7, further comprising:
outputting an indication that a human may not have been detected in response to a determination that a fragment is missing.

### Claim 9 (depends on 7)

The method of claim 7, further comprising:
adjusting parameters to apply to the segmentation map and the boundary map in response to a determination that a fragment is missing.

### Claim 10

A non-transitory medium with instructions stored thereon that, when executed by a processor of an electronic device, cause the processor to perform operations comprising:
acquiring
(i) a segmentation map that is produced for an image that includes a first object and a second object and that indicates whether each pixel in the image is part of the first object or the second object, and
(ii) a boundary map that is produced for the image and that divides the first object from the second object;
identifying, based on the segmentation map and the boundary map, fragments of the first object and the second object that arise from occlusions between the first object and the second object; and
selecting a first group of the fragments for the first object and a second group of the fragments for the second object using one or more bone heatmaps and/or one or more joint heatmaps, such that the first and second objects are associated with no more than a single bone of each type and/or a single joint of each type.

### Claim 11 (depends on 10)

The non-transitory medium of claim 10, wherein the operations further comprise:
transmitting, via a communications interface, identification information that identifies the first object and the second object to a destination external to the electronic device.

### Claim 12 (depends on 10)

The non-transitory medium of claim 10, wherein the one or more bone heatmaps are used to identify connections between the fragments, in order to select the first and second groups of the fragments, and wherein the one or more joint heatmaps are used to assign a joint type to each of the fragments.

### Claim 13 (depends on 12)

The non-transitory medium of claim 12, wherein said selecting is accomplished by clustering the fragments into the first and second groups of the fragments based on the connections identified by the one or more bone heatmaps and then assigning any remaining fragments based on the joint types assigned based on the one or more joint heatmaps, such that there are no common joint types in the first and second groups.

### Claim 14 (depends on 10)

The non-transitory medium of claim 10, wherein the first object is a first human, and wherein the second object is a second human.

### Claim 15 (depends on 10)

The non-transitory medium of claim 10, wherein the image is representative of a frame of a video that is recorded by a camera of the electronic device.

### Claim 16 (depends on 10)

The non-transitory medium of claim 10, wherein the image is representative of a frame of a video that is received from a source external to the electronic device.

### Claim 17

A method comprising:
providing an image that includes a first human and a second human to a computer vision-based segmentation system as input, so as to obtain a segmentation map that indicates whether each pixel in the image is part of the first human or the second human;
generating a boundary map that divides the first human from the second human;
refining the boundary map based on one or more parameters that are selected based on the segmentation map; and
identifying, based on the segmentation map and the boundary map, fragments of the first human and the second human that arise from occlusions between the first and second humans.

### Claim 18 (depends on 17)

The method of claim 17, wherein the boundary map is representative of a matrix of values, each of which is indicative of a likelihood that a corresponding pixel in the image corresponds to a boundary.

### Claim 19 (depends on 18)

The method of claim 18, further comprising:
applying a threshold to the matrix of values, such that each pixel that corresponds to a value above the threshold is assigned a value of one while each pixel that corresponds to a value below the threshold is assigned a value of zero, such that the boundary map is representative of a binary boundary map.

### Claim 20 (depends on 19)

The method of claim 19, further comprising:
adjusting a kernel size of the boundary map to generate closed boundaries with defined lines.
