US 10,872,275 B2Granted

Semantic segmentation based on a hierarchy of neural networks

AssigneeNokia Technologies Oy

InventorTinghuai Wang

Life of this patent

Estimating the term…

Loading patent…

Abstract

Aspects described herein relate to various methods, systems and apparatuses that may improve the accuracy of object classifications and object boundary definitions for a semantic segmentation technique. For example, the semantic segmentation technique may be based on a hierarchy of two or more layers. The two or more layers may include neural networks that analyze image data at different resolution scales. Each layer of the hierarchy may determine object boundary features and object class features. Each layer of the hierarchy may share its object boundary features and/or its object class features with one or more other layers in the hierarchy. In turn, each of other layers of the hierarchy may determine its object boundary features and/or its object class features based on the shared features.

Claims20

Dependent claims

A method comprising:

determining, for a second layer of a hierarchy of neural networks, one or more initial object class features and one or more initial object boundary features;
determining, for the second layer, based on the one or more initial object class features and one or more third layer object class features received from a third layer of the hierarchy, one or more second layer object class features;
determining, for the second layer, based on the one or more initial object boundary features and one or more first layer object boundary features received from a first layer of the hierarchy, one or more second layer object boundary features;
determining, for the second layer, based on the one or more second layer object class features and the one or more second layer object boundary features, one or more fused object class and object boundary features;
determining, for the second layer, based on the one or more initial object class features and the one or more fused object class and object boundary features, segmentation data;
and
determining, based on the segmentation data and additional segmentation data associated with the second layer or the third layer, hierarchy-based segmentation data.

4 more elements

An apparatus comprising:

one or more processors;
and
memory storing executable instructions that, when executed by the one or more processors, cause the apparatus to at least:
determine, for a second layer of a hierarchy of neural networks, one or more initial object class features and one or more initial object boundary features;
determine, for the second layer, based on the one or more initial object class features and one or more third layer object class features received from a third layer of the hierarchy, one or more second layer object class features;
determine, for the second layer, based on the one or more initial object boundary features and one or more first layer object boundary features received from a first layer of the hierarchy, one or more second layer object boundary features;
determine, for the second layer, based on the one or more second layer object class features and the one or more second layer object boundary features, one or more fused object class and object boundary features;
determine, for the second layer, based on the one or more initial object class features and the one or more fused object class and object boundary features, segmentation data;
and
determine, based on the segmentation data and additional segmentation data associated with the second layer or the third layer, hierarchy-based segmentation data.

7 more elements

One or more non-transitory computer-readable media storing executable instructions that, when executed, cause an apparatus to at least:

determine, for a second layer of a hierarchy of neural networks, one or more initial object class features and one or more initial object boundary features;
determine, for the second layer, based on the one or more initial object class features and one or more third layer object class features received from a third layer of the hierarchy, one or more second layer object class features;
determine, for the second layer, based on the one or more initial object boundary features and one or more first layer object boundary features received from a first layer of the hierarchy, one or more second layer object boundary features;
determine, for the second layer, based on the one or more second layer object class features and the one or more second layer object boundary features, one or more fused object class and object boundary features;
determine, for the second layer, based on the one or more initial object class features and the one or more fused object class and object boundary features, segmentation data;
and
determine, based on the segmentation data and additional segmentation data associated with the second layer or the third layer, hierarchy-based segmentation data.

4 more elements

Dependencies are inferred from the claim text.

Facts

Application: 16/361,433Filed 22 Mar 2019
Published: US 2020/0302239 A124 Sep 2020
Granted: 22 Dec 2020
Term ends: Not estimatedThe source records don’t support an estimate.

Classification15 codes

G06T 7/12
Segmentation; Edge detection: edge-based segmentation
G06F 18/217
Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation: validation; Performance…
G06N 3/045
Architecture, e.g. interconnection topology: combinations of networks
G06N 3/0464
Architecture, e.g. interconnection topology: convolutional networks [CNN, ConvNet]

Family

US 2020/0302239 A1Pre-grant publication · published 24 Sep 2020
US 10,872,275 B2This patent

From the US filing history. Foreign family members aren’t listed.

Timeline

Citations

Open elsewhere

Building an agent or pulling many patents? Use the API or MCP server instead of scraping pages. Get an API key