# US 2019/0356868 A1 · Clothes and accessories fitting method, display system and computer-readable recording medium thereof

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0356868 A1
- Document ID: US20190356868A1
- Status: Published application
- Kind: A1
- Application number: 16/116,916
- Filed: 2018-08-30
- Published: 2019-11-21
- Earliest claimed: 2018-05-16
- Assignees: Wistron Corporation
- Applicants: Wistron Corporation
- Inventors: Bo-Yi Wu; Hui-Chen Lin; Ge-Yi Lin
- Web page: https://patentgrep.com/patent/US20190356868A1

## CPC

- G06T 7/11: Region-based segmentation
- H04N 5/272: Means for inserting a foreground image in a background image, i.e. inlay, outlay
- G06T 7/194: involving foreground-background segmentation
- G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)}
- H04N 5/2226: {Determination of depth image, e.g. for foreground/background separation (determining depth by image analysis in general G06T7/50; segmentation by image analysis in general G06T7/10)}
- H04N 9/8205: {involving the multiplexing of an additional signal and the colour video signal}
- H04N 23/611: where the recognised objects include parts of the human body
- H04N 23/63: by using electronic viewfinders
- H04N 23/64: {Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image}
- H04N 23/667: Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
- G06T 2219/2012: Colour editing, changing, or manipulating; Use of colour codes
- G06T 2219/2024: Style variation
- H04N 5/9201: {involving the multiplexing of an additional signal and the video signal}
- H04N 23/57: Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
- H04N 2005/2726: {for simulating a person's appearance, e.g. hair style, glasses, clothes}
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A clothes and accessories fitting method, a display system and a computer-readable recording medium thereof are provided, where the method is applicable to a display system having an image capturing device and a screen and includes the following steps. Images of a user wearing a first apparel appearing in front of the screen are continuously or intermittently captured by using the image capturing device. When a first image of the user in a particular pose is determined to be included in the images, the first image is shot by using the image capturing device. A foreground region and a background region in the first image are segmented, and the background region is processed to generate a processed background region. A first preview image is generated according to the foreground region and the processed background region and displayed on the screen.

## Claims

### Claim 1

A clothes and accessories fitting method, applicable to a display system having an image capturing device and a screen, comprising:
capturing a plurality of images of a user wearing a first apparel appearing in front of the screen continuously or intermittently by using the image capturing device;
in response that a first image of the user in a particular pose is determined to be included in the images, shooting the first image by using the image capturing device;
segmenting a foreground region and a background region in the first image, and processing the background region to generate a processed background region; and
generating a first preview image according to the foreground region and the processed background region, and displaying the preview image on the screen.

### Claim 2 (depends on 1)

The method according to claim 1, wherein during the step of capturing the images of the user wearing the first apparel appearing in front of the screen continuously or intermittently by using the image capturing device, the method further comprises:
presetting the display system to enter a mirror mode so as to allow the images captured by the image capturing device to be displayed on the screen in real time.

### Claim 3 (depends on 2)

The method according to claim 2, wherein during the step of capturing the images of the user wearing the first apparel appearing in front of the screen continuously or intermittently by using the image capturing device, the method further comprises:
determining whether there exists a potential user in the images; and
in response that the potential user is determined to be existed in the images, setting the display system to enter a video mode so as to allow the images captured by the image capturing device to be displayed on the screen and recording the images to generate a recorded video.

### Claim 4 (depends on 3)

The method according to claim 3 further comprising:
in response that the potential user in the images is determined to leave, setting the display system to leave the video mode and to return to the mirror mode and deleting the recorded video.

### Claim 5 (depends on 3)

The method according to claim 3 further comprising:
in response that the potential user is determined to be an authorized user of the display system, setting the display system to leave the video mode and storing the recorded video in a database.

### Claim 6 (depends on 1)

The method according to claim 1, wherein before the first image of the user in the particular pose is determined to be included in the images, the method further comprises:
setting the display system to enter a try-on mode.

### Claim 7 (depends on 6)

The method according to claim 6, wherein while the display system is in the try-on mode, after the step of generating the first preview image, the method further comprises:
setting the display system to leave the try-on mode and displaying the first preview image on the screen.

### Claim 8 (depends on 7)

The method according to claim 7, wherein while the display system is in the try-on mode, after the step of generating the first preview image, the method further comprises:
capturing at least one other first image of the user wearing the first apparel in other poses by using the image capturing device; and
processing the at least one other first image to generate at least one other first preview image.

### Claim 9 (depends on 8)

The method according to claim 8, wherein after the step of generating the first preview image and the at least one other first preview image, the method further comprises:
storing the first preview image and the at least one other first preview image in a database; and
setting the display system to leave the try-on mode and enter a lookbook mode.

### Claim 10 (depends on 9)

The method according to claim 9, wherein the database further comprises at least one preview image corresponding to at least one other apparel, and wherein in response that the display system enters the lookbook mode, the method further comprises:
detecting a selection operation performed by the user on a designated pose among a plurality of poses; and
displaying a plurality of designated preview images corresponding to the designated pose on the screen, wherein each of the designated preview images is associated with a processed image of the user wearing a different apparel in the same designated pose.

### Claim 11 (depends on 1)

The method according to claim 1, wherein the step of segmenting the foreground region and the background region in the first image comprises:
performing facial detection on the first image to define a facial region;
defining a foreground segmented region in the first image according to the facial region;
obtaining a depth image corresponding to the first image;
performing binarization on the depth image to accordingly generate a plurality of foreground depth pixels;
defining and setting a plurality of foreground plane pixels corresponding to the foreground depth pixels in the foreground segmented region as the foreground region; and
setting a region outside the foreground region in the first image as the foreground region.

### Claim 12 (depends on 11)

The method according to claim 11, wherein the step of defining the foreground segmented region in the first image according to the facial region comprises:
defining a head region of the user according to the facial region;
calculating a height of the user according to the head region; and
defining a boundary of the foreground segmented region according to the facial region and the height of the user.

### Claim 13 (depends on 11)

The method according to claim 11, wherein the step of performing binarization on the depth image to accordingly generate the foreground depth pixels comprises:
performing binarization on the depth image to generate a plurality of foreground pixels; and
performing erosion, dilation, and blur processing on the foreground pixels to generate the foreground depth pixels.

### Claim 14 (depends on 13)

The method according to claim 13, wherein the step of generating the foreground depth pixels further comprises:
setting a background depth pixel neighboring to and having a similar pixel value to the foreground depth pixels as one of the foreground depth pixels.

### Claim 15 (depends on 13)

The method according to claim 13, wherein the step of generating the foreground depth pixels further comprises:
obtaining a distance range of the user with respect to the image capturing device;
obtaining a depth information range corresponding to the distance range according to a depth-distance relationship table; and
filtering out the foreground depth pixels with depth information being greater than the depth information range.

### Claim 16 (depends on 1)

The method according to claim 1, wherein the step of processing the background region to generate the processed background region comprises:
performing blur processing on the background region to generate the processed background region.

### Claim 17 (depends on 1)

The method according to claim 1, wherein the step of processing the background region to generate the processed background region comprises:
setting a preset background as the processed background region.

### Claim 18 (depends on 1)

The method according to claim 1, wherein the step of processing the background region to generate the processed background region comprises:
obtaining related information of the first apparel according to an identification code of the first apparel and thereby obtaining a color shade of the first apparel;
generating a corresponding background according to the color shade of the first apparel;
setting the corresponding background as the processed background.

### Claim 19

A display system comprising:
an image capturing device, configured to capture images;
a screen, configured to display images, wherein a capturing direction of the image capturing device is the same as a display direction of the screen;
a data storage device, configured to store data; and
a processor, coupled to the image capturing device, the screen, and the data storage device, and configured to:
capture a plurality of images of a user wearing a first apparel appearing in front of the screen continuously or intermittently by using the image capturing device;
in response that a first image of the user in a particular pose is determined to be included in the images of the user wearing the first apparel, shoot the first image by using the image capturing device;
segment a foreground region and a background region in the first image, and process the background region to generate a processed background region; and
generate a first preview image according to the foreground region and the processed background region, and display the preview image on the screen.

### Claim 20

A non-transitory computer-readable recording medium, recording computer programs to be loaded and executed by a processor of a display system having an image capturing device and a screen to perform steps of:
capturing a plurality of images of a user wearing a first apparel appearing in front of the screen continuously or intermittently by using the image capturing device;
in response that a first image of the user in a particular pose is determined to be included in the images, shooting the first image by using the image capturing device;
segmenting a foreground region and a background region in the first image, and processing the background region to generate a processed background region; and
generating a first preview image according to the foreground region and the processed background region, and displaying the preview image on the screen.
