# US 2025/0148748 A1 · Method and electronic device for interactive image segmentation

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2025/0148748 A1
- Document ID: US20250148748A1
- Status: Published application
- Kind: A1
- Application number: 19/016,900
- Filed: 2025-01-10
- Published: 2025-05-08
- Earliest claimed: 2022-07-12
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung electronics Co., LTD.
- Inventors: Praful MATHUR; Shashi Kumar Parwani; Darshana Venkatesh Murthy; Roopa Kotiganahally Sheshadri; Aman Sharma; Veerendra K Shetty; Sunmin Park
- Web page: https://patentgrep.com/patent/US20250148748A1

## CPC

- G06T 7/11: Region-based segmentation
- G06V 10/26: Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
- G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence)
- G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour
- G06V 10/235: {based on user input or interaction}
- G06V 10/32: Normalisation of the pattern dimensions
- G06V 10/44: Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
- G06V 10/56: relating to colour
- G06V 10/82: using neural networks
- G06F 40/279: Recognition of textual entities
- G06T 2207/10024: Color image
- G06T 2207/20076: Probabilistic image processing
- G06T 2207/20081: Training; Learning
- G06T 2207/20084: Artificial neural networks [ANN]
- G06T 2207/20104: Interactive definition of region of interest [ROI]
- G06T 2207/20132: Image cropping
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

Provided are a method and electronic device for interactive image segmentation. The method includes receiving one or more user inputs for segmenting at least one object from among a plurality of objects in an image. The method includes generating a unified guidance map indicates the at least one object to be segmented based on the one or more user inputs. The method includes generating a complex supervision image based on the unified guidance map. The method includes segmenting the at least one object from the image by passing the image, the complex supervision image and the unified guidance map through an adaptive Neural Network (NN) model. The method includes storing the at least one segmented object from the image. Further, the method include configuring the parameters of an adaptive Neural Network based on color complexity analysis, edge complexity analysis and geometry complexity analysis.

## Claims

### Claim 1

A method for interactive image segmentation by an electronic device, the method comprising: receiving one or more user inputs for segmenting at least one object from among a plurality of objects in an image;
generating a unified guidance map that indicates the at least one object to be segmented based on the one or more user inputs;
generating a complex supervision image based on the unified guidance map;
segmenting the at least one object from the image by inputting the image, the complex supervision image and the unified guidance map into an adaptive Neural Network (NN) model; and
storing the at least one segmented object from the image.

### Claim 2 (depends on 1)

The method as claimed in claim 1, wherein the generating the unified guidance map comprises:
extracting input data based on the one or more user inputs;
obtaining one or more guidance maps corresponding to the one or more user inputs based on the input data; and
generating the unified guidance map by concatenating the one or more guidance maps.

### Claim 3 (depends on 2)

The method as claimed in claim 2, wherein the obtaining the one or more guidance maps comprises:
based on the input data comprising one or more set of coordinates, obtaining traces of the one or more user inputs on the image using the input data; and
encoding the traces into the one or more guidance maps, and
wherein the traces represent user interaction locations on the image.

### Claim 4 (depends on 2)

The method as claimed in claim 2, wherein the obtaining the one or more guidance maps comprises:
based on the input data comprising text indicating the at least one object in the image, determining a segmentation mask based on a category of the text using an instance model; and
converting the segmentation mask into the one or more guidance maps.

### Claim 5 (depends on 4)

The method as claimed in claim 4, wherein the determining the segmentation mask comprises:
based on the input data comprising an audio, converting the audio into the text; and
determining the segmentation mask based on the category of the text using the instance model.

### Claim 6 (depends on 1)

The method as claimed in claim 1, wherein the generating the complex supervision image comprises:
determining a plurality of complexity parameters comprising at least one of a color complexity, an edge complexity or a geometry map of the at least one object to be segmented; and
generating the complex supervision image by concatenating a weighted low frequency image obtained using the color complexity and the unified guidance map, a weighted high frequency image obtained using the edge complexity and the unified guidance map, and the geometry map.

### Claim 7 (depends on 6)

The method as claimed in claim 6, wherein the determining the color complexity of the at least one object comprises:
obtaining a low frequency image by inputting the image into a low pass filter;
determining a weighted map by normalizing the unified guidance map;
determining a weighted low frequency image by convolving the low frequency image with the weighted map;
determining a standard deviation of the weighted low frequency image;
determining whether the standard deviation of the weighted low frequency image is greater than a first threshold; and
performing one of:
detecting that the color complexity is high, based on the standard deviation of the weighted low frequency image being greater than the first threshold, and
detecting that the color complexity is low, based on the standard deviation of the weighted low frequency image being less than or equal to the first threshold.

### Claim 8 (depends on 6)

The method as claimed in claim 6, wherein the determining the edge complexity of the at least one object comprises:
obtaining a high frequency image by inputting the image into a high pass filter;
determining a weighted map by normalizing the unified guidance map;
determining a weighted high frequency image by convolving the high frequency image with the weighted map;
determining a standard deviation of the weighted high frequency image for analyzing the edge complexity;
determining whether the standard deviation of the weighted high frequency image is greater than a second threshold; and
performing one of:
detecting that the edge complexity is high, based on the standard deviation of the weighted high frequency image being greater than the second threshold, and
detecting that the edge complexity is low, based on the standard deviation of the weighted high frequency image being less than or equal to the second threshold.

### Claim 9 (depends on 6)

The method as claimed in claim 6, wherein the determining the geometry map of the at least one object comprises:
identifying a color at a location on the image;
tracing the color within a reference range of color at the location;
obtaining the geometry map comprising a union of the traced color with an edge map of the at least one object; and
estimating a span of the at least one object by determining a size of a bounding box of the at least one object in the geometry map, and
wherein the span corresponds to a larger side of the bounding box in a rectangle shape.

### Claim 10 (depends on 1)

The method as claimed in claim 1, wherein the segmenting the at least one object from the image comprises:
determining optimal scales for the adaptive NN model based on a relationship between a receptive field of the adaptive NN model and a span of the at least one object;
determining an optimal number of layers for the adaptive NN model based on a color complexity;
determining an optimal number of channels for the adaptive NN model based on an edge complexity;
configuring the adaptive NN model based on the optimal scales, the optimal number of layers, and the optimal number of channels; and
segmenting the at least one object from the image by inputting the image, the complex supervision image, and the unified guidance map through the configured adaptive NN model.

### Claim 11 (depends on 10)

The method as claimed in claim 10, wherein the determining the optimal scales comprises:
downscaling the image by a factor of two until the span of matches to the receptive field; and
determining the optimal scales for the adaptive NN model based on a number of times the image has been downscaled to match the span with the receptive field.

### Claim 12 (depends on 10)

The method as claimed in claim 10, wherein the determining the optimal number of layers comprises:
performing one of:
selecting a default number of layers as the optimal number of layers based on detecting a first color complexity in the image, and
adding a reference layer offset value with the default number of layers for obtaining the optimal number of layers based on detecting a second color complexity, and
wherein the first color complexity is lower than the second color complexity.

### Claim 13 (depends on 10)

The method as claimed in claim 10, wherein the determining the optimal number of channels comprises:
performing one of:
selecting a default number of channels as the optimal number of channels based on detecting a first edge complexity, and
adding a reference channel offset value with the default number of channels for obtaining the optimal number of channels based on detecting a second edge complexity, and
wherein the first edge complexity is lower than the second edge complexity.

### Claim 14

A method for encoding different types of user interactions by an electronic device, the method comprising:
detecting a plurality of user inputs performed on an image;
obtaining a plurality of guidance maps by converting each of the plurality of user inputs to one of the plurality of guidance maps based on a type of the respective user input;
unifying the plurality of guidance maps to generate a unified guidance map representing a unified feature space;
determining an object complexity based on the unified guidance map and the image; and
inputting the object complexity and the image to an interactive segmentation engine.

### Claim 15 (depends on 14)

The method as claimed in claim 14, wherein the type of the user inputs is at least one of a touch, a contour, a scribble, a stroke, text, an audio, an eye gaze, or an air gesture.
