# US 2021/0279525 A1 · Hierarchy-preserving learning for multi-label classification

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0279525 A1
- Document ID: US20210279525A1
- Status: Published application
- Kind: A1
- Application number: 17/192,761
- Filed: 2021-03-04
- Published: 2021-09-09
- Earliest claimed: 2020-03-05
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Company, Ltd.
- Inventors: Palash Goyal; Divya Choudhary; Saurabh Sahu; Shalini Ghosh
- Web page: https://patentgrep.com/patent/US20210279525A1

## CPC

- G06N 5/022: Knowledge engineering; Knowledge acquisition
- G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting
- G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques}
- G06F 18/2431: Multiple classes
- G06F 18/24323: {Tree-organised classifiers}
- G06N 20/00: Machine learning
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting
- G06V 10/776: Validation; Performance evaluation
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A method implemented by one or more computing systems includes accessing a set of content objects, in which each content object of the set of content objects is pre-labeled with concepts of a plurality of concepts organized according to a hierarchical relationship. The method further includes training, by a machine-learning model, a classification model for classifying content objects within the set of content objects. Training the classification model includes determining, for each object, a plurality of classification values corresponding to the plurality of concepts, calculating a loss for each of the plurality of classification values based on the pre-labeled concepts associated with the object, utilizing a hierarchical constraint loss function to calculate a maximum loss based on the calculated loss for each of the plurality of classification values, and updating the classification model based on the hierarchical constraint loss function until the maximum loss satisfies a predetermined criterion.

## Claims

### Claim 1

A method comprising, by one or more computing systems:
accessing a set of content objects, wherein each content object of the set of content objects is pre-labeled with one or more concepts of a plurality of concepts, and wherein the plurality of concepts are organized according to a hierarchical relationship; and
training, by a machine-learning model, a classification model for classifying content objects within the set of content objects, wherein training the classification model comprises:
determining, for each object, a plurality of classification values corresponding to the plurality of concepts;
calculating a loss for each of the plurality of classification values based on the pre-labeled concepts associated with the object;
utilizing a hierarchical constraint loss function to calculate a maximum loss based on the calculated loss for each of the plurality of classification values; and
updating the classification model based on the hierarchical constraint loss function until the maximum loss satisfies a predetermined criterion.

### Claim 2 (depends on 1)

The method of claim 1, wherein the calculated loss for each of the plurality of classification values comprises a first calculated loss for a parent concept of the plurality of concepts and a second calculated loss for a child concept of the plurality of concepts.

### Claim 3 (depends on 2)

The method of claim 2, wherein the hierarchical constraint loss function comprises one or more constraints configured to limit the second calculated loss to a value less than or equal to that of the first calculated loss.

### Claim 4 (depends on 2)

The method of claim 2, wherein the hierarchical constraint loss function is bounded around a 0-1 loss function.

### Claim 5 (depends on 2)

The method of claim 2, wherein training, by the machine-learning model, the classification model comprises training the classification model utilizing content objects of the set of content objects corresponding to the parent concept in accordance with the predetermined criteria prior to utilizing content objects of the set of content objects corresponding to the child concept in accordance with the predetermined criteria.

### Claim 6 (depends on 1)

The method of claim 1, further comprising:
prior to utilizing the hierarchical constraint loss function to calculate the maximum loss, deriving the hierarchical constraint loss function based on one or more base loss functions.

### Claim 7 (depends on 6)

The method of claim 6, wherein, subsequent to deriving the hierarchical constraint loss function, the method further comprising:
sorting the respective losses for each of the plurality of classification values in order of increasing loss value; and
selecting an initial K concepts corresponding to an initial K losses of the sorted respective losses, such that a cumulative sum of the initial K losses is greater than a sum of a predetermined threshold loss value and 1−K, wherein K is a hyperparameter.

### Claim 8

One or more computing devices, comprising:
one or more non-transitory computer-readable storage media including instructions; and
one or more processors coupled to the one or more storage media, the one or more processors configured to execute the instructions to:
access a set of content objects, wherein each content object is pre-labeled with one or more concepts of a plurality of concepts, and wherein the plurality of concepts are organized according to a hierarchical relationship; and
train, by a machine-learning model, a classification model for classifying content objects within the set of content objects, wherein training the classification model comprises:
determine, for each object, a plurality of classification values corresponding to the plurality of concepts;
calculate a loss for each of the plurality of classification values based on the pre-labeled concepts associated with the object;
utilize a hierarchical constraint loss function to calculate a maximum loss based on the calculated loss for each of the plurality of classification values; and
update the classification model based on the hierarchical constraint loss function until the maximum loss satisfies a predetermined criterion.

### Claim 9 (depends on 8)

The one or more computing devices of claim 8, wherein the calculated loss for each of the plurality of classification values comprises a first calculated loss for a parent concept of the plurality of concepts and a second calculated loss for a child concept of the plurality of concepts.

### Claim 10 (depends on 9)

The one or more computing devices of claim 9, wherein the hierarchical constraint loss function comprises one or more constraints configured to limit the second calculated loss to a value less than or equal to that of the first calculated loss.

### Claim 11 (depends on 9)

The one or more computing devices of claim 9, wherein the hierarchical constraint loss function is bounded around a 0-1 loss function.

### Claim 12 (depends on 9)

The one or more computing devices of claim 9, wherein the instructions to train, by the machine-learning model, the classification model further comprises instructions to train the classification model utilizing content objects of the set of content objects corresponding to the parent concept in accordance with the predetermined criteria prior to utilizing content objects of the set of content objects corresponding to the child concept in accordance with the predetermined criteria.

### Claim 13 (depends on 8)

The one or more computing devices of claim 8, wherein, prior to utilizing the hierarchical constraint loss function to calculate the maximum loss, the instructions further comprises instructions to:
derive the hierarchical constraint loss function based on one or more base loss functions.

### Claim 14 (depends on 13)

The one or more computing devices of claim 13, wherein, subsequent to deriving the hierarchical constraint loss function, the instructions further comprises instructions to:
sort the respective losses for each of the plurality of classification values in order of increasing loss value; and
select an initial K concepts corresponding to an initial K losses of the sorted respective losses, such that a cumulative sum of the initial K losses is greater than a sum of a predetermined threshold loss value and 1−K, wherein K is a hyperparameter.

### Claim 15

A non-transitory computer-readable medium comprising instructions that, when executed by one or more processors of one or more computing devices, cause the one or more processors to:
access a set of content objects, wherein each content object is pre-labeled with one or more concepts of a plurality of concepts, and wherein the plurality of concepts are organized according to a hierarchical relationship; and
train, by a machine-learning model, a classification model for classifying content objects within the set of content objects, wherein training the classification model comprises:
determine, for each object, a plurality of classification values corresponding to the plurality of concepts;
calculate a loss for each of the plurality of classification values based on the pre-labeled concepts associated with the object;
utilize a hierarchical constraint loss function to calculate a maximum loss based on the calculated loss for each of the plurality of classification values; and
update the classification model based on the hierarchical constraint loss function until the maximum loss satisfies a predetermined criterion.

### Claim 16 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein the calculated loss for each of the plurality of classification values comprises a first calculated loss for a parent concept of the plurality of concepts and a second calculated loss for a child concept of the plurality of concepts.

### Claim 17 (depends on 16)

The non-transitory computer-readable medium of claim 16, wherein the hierarchical constraint loss function comprises one or more constraints configured to limit the second calculated loss to a value less than or equal to that of the first calculated loss.

### Claim 18 (depends on 16)

The non-transitory computer-readable medium of claim 16, wherein the hierarchical constraint loss function is bounded around a 0-1 loss function.

### Claim 19 (depends on 15)

The non-transitory computer-readable medium of claim 15, wherein, prior to utilizing the hierarchical constraint loss function to calculate the maximum loss, the instructions further comprises instructions to:
derive the hierarchical constraint loss function based on one or more base loss functions.

### Claim 20 (depends on 19)

The non-transitory computer-readable medium of claim 19, wherein, subsequent to deriving the hierarchical constraint loss function, the instructions further comprises instructions to:
sort the respective losses for each of the plurality of classification values in order of increasing loss value; and
select an initial K concepts corresponding to an initial K losses of the sorted respective losses, such that a cumulative sum of the initial K losses is greater than a sum of a predetermined threshold loss value and 1−K, wherein K is a hyperparameter.