# US 10,872,420 B2 · Electronic device and method for automatic human segmentation in image

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,872,420 B2
- Document ID: US10872420B2
- Status: Granted
- Kind: B2
- Application number: 16/126,636
- Filed: 2018-09-10
- Published: 2019-03-14
- Granted: 2020-12-22
- Earliest claimed: 2017-09-08
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Alok Shankarlal Shukla; Mahesh Putane Jagadeeshrao; Nitin Kumar Singh; Divay Bhutani; Nitin Kamboj; Manoj Kumar Marram Reddy; Prasanna Kumar Avanigadda; Sanket Sambhajirao Deshmukh; Insung Hwang
- Web page: https://patentgrep.com/patent/US10872420B2

## CPC

- G06T 7/194: involving foreground-background segmentation
- G06F 18/253: {of extracted features}
- G06N 3/02: Neural networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 20/00: Machine learning
- G06T 5/20: using local operators
- G06T 5/70: Denoising; Smoothing
- G06T 5/73: Deblurring; Sharpening
- G06T 7/11: Region-based segmentation
- G06T 7/521: from laser ranging, e.g. using interferometry; from the projection of structured light
- G06V 10/806: {of extracted features}
- G06V 10/82: using neural networks
- G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V 40/161: {Detection; Localisation; Normalisation}
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06T 2207/30196: Human being; Person
- G06T 2207/30201: Face
- G06V 10/267: {by performing operations on regions, e.g. growing, shrinking or watersheds}
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

An electronic device and a method for human segmentation in an image are provided. The method includes obtaining the image comprising at least one human, performing face detection to detect the human in the image, generating a first human segmentation from the image by comparing a pre-defined full human body template and the detected human, generating a second human segmentation from the image, by inputting the image to an artificial intelligence trained model and obtaining a composite human segmentation representing the detected human by using both of the first human segmentation and the second human segmentation.

## Claims

### Claim 1

A method for human segmentation in an image by an electronic device, the method comprising:
obtaining the image comprising a human;
performing face detection to detect the human in the image;
generating a first human segmentation, without using a machine learning algorithm or neural network model, from the image by comparing a pre-defined full human body template and the detected human;
generating a second human segmentation from the image by inputting the image to an artificial intelligence trained model;
obtaining a composite human segmentation representing the detected human by using both of the first human segmentation and the second human segmentation; and
applying a bokeh effect in the image based on the composite human segmentation by:
blurring the image, and
blending the blurred image and information about the human obtained based on the composite human segmentation,
wherein each of the first human segmentation and the second human segmentation includes information indicating an outline of a shape of the human, and
wherein the generating of the second human segmentation from the image comprises:
reducing a resolution of the image;
inputting the image with the reduced resolution to the artificial intelligence trained model; and
generating the second human segmentation from the image based on an output of the artificial intelligence trained model.

### Claim 2 (depends on 1)

The method of claim 1, further comprising:
determining a template coefficient based on the face detection,
wherein at least one of a size, direction, or a location of the pre-defined full human body template is adjusted based on the template coefficient to generate the first human segmentation.

### Claim 3 (depends on 1)

The method of claim 1, wherein the first human segmentation is generated by:
aligning the pre-defined full human body template to the human detected in the image;
selecting a part of the pre-defined full human body template; and
refining the selected part of the pre-defined full human body template.

### Claim 4 (depends on 1)

The method of claim 1, wherein the image is a live preview image captured by a camera of the electronic device.

### Claim 5 (depends on 1)

The method of claim 1, further comprising refining the first human segmentation based on the image.

### Claim 6 (depends on 5)

The method of claim 5, wherein the refining of the first human segmentation based on the image is provided by adjusting a boundary of the first human segmentation using a color similarity based refinement method.

### Claim 7 (depends on 1)

The method of claim 1, further comprising:
obtaining a structured light data; and
segmenting each human in the image separately based on the structured light data and the composite human segmentation.

### Claim 8 (depends on 1)

The method of claim 1, wherein the artificial intelligence trained model is a trained model which is trained using at least one of a deep neural network (DNN), machine learning, neural network, deep learning, or classification algorithm.

### Claim 9

An electronic device for human segmentation in an image, the electronic device comprising:
a display;
a memory storing at least one instruction; and
a processor configured to execute the at least one instruction to:
obtain the image comprising a human;
perform face detection to detect the human in the image;
generate a first human segmentation, without using a machine learning algorithm or neural network model, from the image by comparing a pre-defined full human body template and the detected human;
generate a second human segmentation from the image by inputting the image to an artificial intelligence trained model;
obtain a composite human segmentation representing the detected human by using both of the first human segmentation and the second human segmentation; and
apply a bokeh effect in the image based on the composite human segmentation by:
blurring the image, and
blending the blurred image and information about the human obtained based on the composite human segmentation,
wherein each of the first human segmentation and the second human segmentation includes information indicating an outline of a shape of the human, and
wherein the processor, to generate the second human segmentation from the image, is further configured to execute the at least one instruction to:
reduce a resolution of the image;
input the image with the reduced resolution to the artificial intelligence trained model; and
generate the second human segmentation from the image based on an output of the artificial intelligence trained model.

### Claim 10 (depends on 9)

The electronic device of claim 9,
wherein the processor is further configured to execute the at least one instruction to determine a template coefficient based on the face detection, and
wherein at least one of a size, direction, or a location of the pre-defined full human body template is adjusted based on the template coefficient to generate the first human segmentation.

### Claim 11 (depends on 9)

The electronic device of claim 9, wherein the first human segmentation is generated by:
aligning the pre-defined full human body template to the human detected in the image;
selecting a part of the pre-defined full human body template; and
refining the selected part of the pre-defined full human body template.

### Claim 12 (depends on 9)

The electronic device of claim 9, wherein the image is a live preview image captured by a camera of the electronic device.

### Claim 13 (depends on 9)

The electronic device of claim 9, wherein the processor is further configured to execute the at least one instruction to refine the first human segmentation based on the image by adjusting a boundary of the first human segmentation using a color similarity based refinement method.

### Claim 14 (depends on 9)

The electronic device of claim 9, wherein the processor is further configured to execute the at least one instruction to:
obtain a structured light data; and
segment each human in the image separately based on the structured light data and the composite human segmentation.

### Claim 15 (depends on 9)

The electronic device of claim 9, wherein the artificial intelligence trained model is a trained model which is trained using at least one of a deep neural network (DNN), machine learning, neural network, deep learning, or classification algorithm.

### Claim 16

A non-transitory computer-readable recording medium having recorded thereon a program which, when executed by a computer, performs a method of human segmentation in an image, by a device, the method comprising:
obtaining the image comprising a human;
performing face detection to detect the human in the image;
generating a first human segmentation, without using a machine learning algorithm or neural network model, from the image by comparing a pre-defined full human body template and the detected human;
generating a second human segmentation from the image by inputting the image to an artificial intelligence trained model;
obtaining a composite human segmentation representing the detected human by using both of the first human segmentation and the second human segmentation; and
applying a bokeh effect in the image based on the composite human segmentation by:
blurring the image, and
blending the blurred image and information about the human obtained based on the composite human segmentation,
wherein each of the first human segmentation and the second human segmentation includes information indicating an outline of a shape of the human, and
wherein the generating of the second human segmentation from the image comprises:
reducing a resolution of the image;
inputting the image with the reduced resolution to the artificial intelligence trained model; and
generating the second human segmentation from the image based on an output of the artificial intelligence trained model.