# US 2021/0067692 A1 · Mobile terminal device that generates picture where person's eye direction in shot image has been modified

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0067692 A1
- Document ID: US20210067692A1
- Status: Published application
- Kind: A1
- Application number: 17/002,222
- Filed: 2020-08-25
- Published: 2021-03-04
- Earliest claimed: 2019-09-03
- Assignees: KYOCERA Document Solutions Inc.
- Applicants: KYOCERA Document Solutions Inc.
- Inventors: Philogene Kyle DIMPAS
- Web page: https://patentgrep.com/patent/US20210067692A1

## CPC

- H04N 5/772: {the recording apparatus and the television camera being placed in the same enclosure}
- G06V 10/24: Aligning, centring, orientation detection or correction of the image
- G06V 20/20: in augmented reality scenes
- G06V 40/165: {using facial parts and geometric relationships}
- G06V 40/19: Sensors therefor
- G06V 40/193: {Preprocessing; Feature extraction}
- H04N 5/76: Television signal recording
- H04N 23/51: Housings
- H04N 23/54: Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
- H04N 23/57: Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
- H04N 23/611: where the recognised objects include parts of the human body
- H04N 23/631: {Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters}
- H04N 23/64: {Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image}
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

A mobile terminal device includes: a storage device that stores shot images taken by a camera; a face recognizer that recognizes a person's face image contained in a shot image selected from among the shot images stored in the storage device; a determiner that determines whether or not the person's face image recognized by the face recognizer is a camera-eye-aligned image where an eye direction of the person is aligned with the camera; an image modifier that modifies an image of an eye portion in the face image determined not to be a camera-eye-aligned image by the determiner to an image where the eye direction is aligned with the camera; and a controller that allows the storage device to store the shot image modified by the image modifier.

## Claims

### Claim 1

A mobile terminal device comprising:
a housing;
a display device disposed on a particular one of a plurality of outside surfaces of the housing;
a camera disposed at a site of the particular surface of the housing other than the display device;
an image storage device that stores shot images taken by the camera; and
a control device that includes a processor and functions, through the processor executing a control program, as:
a face recognizer that recognizes a person's face image contained in a shot image selected from among the shot images stored in the image storage device;
a determiner that determines whether or not the person's face image recognized by the face recognizer is a camera-eye-aligned image where an eye direction of the person is aligned with the camera;
an image modifier that modifies an image of an eye portion in the face image determined not to be a camera-eye-aligned image by the determiner to an image where the eye direction is aligned with the camera; and
a controller that allows the image storage device to store the shot image modified by the image modifier.

### Claim 2 (depends on 1)

The mobile terminal device according to claim 1,
further comprising an operation device capable of being operated by a user,
wherein the control device further functions, through the processor executing the control program, as a display controller that allows the display device to display the shot image selected according to a selecting operation made on the operation device by the user,
the display controller allows, in the shot image displayed on the display device, an eye portion in the face image determined not to be a camera-eye-aligned image by the determiner to be displayed in a representation indicating that the eye portion is a modification target region,
in presence of an instruction to modify the modification target region in the shot image displayed on the display device, the image modifier modifies an image of the eye portion which is the modification target region to an image where the eye direction is aligned with the camera, and
in absence of the instruction to modify the modification target region in the shot image displayed on the display device, the image modifier avoids modification of the image of the eye portion which is the modification target region.

### Claim 3 (depends on 2)

The mobile terminal device according to claim 2, wherein
the control device further functions, through the processor executing the control program, as a picture-taker identifier that identifies, among persons contained in the selected shot image, a picture-taker having taken the shot image,
when the modification target region for the picture-taker identified by the picture-taker identifier is displayed in the shot image displayed on the display device, the image modifier modifies an image of an eye portion of the picture-taker, which is the modification target region for the picture-taker, to an image where the eye direction is aligned with the camera, and
in presence of an instruction to modify a modification target region for any person other than the picture-taker, the image modifier modifies an image of an eye portion of the person other than the picture-taker, which is the modification target region for the person.

### Claim 4 (depends on 3)

The mobile terminal device according to claim 3, wherein
when a number of persons' face images in the selected shot image recognized by the face recognizer is one, the picture-taker identifier identifies a person having the face image as a picture-taker, and
when the number of persons' face images in the selected shot image recognized by the face recognizer is plural, the picture-taker identifier identifies as a picture-taker a person who has a face image located at an end side of the shot image and having a largest facial region in the plurality of face images or a person who has a face image located at an end side of the shot image and has an arm extending to the end side of the shot image.

### Claim 5 (depends on 4)

The mobile terminal device according to claim 4, wherein when the picture-taker identifier fails to identify a picture-taker in the selected shot image containing a plurality of persons' face images recognized by the face recognizer, the picture-taker identifier identifies as a picture-taker a person specified in the selected shot image according to a specification operation made on the operation device by the user.

### Claim 6 (depends on 3)

The mobile terminal device according to claim 3, wherein the picture-taker identifier identifies as a picture-taker a person specified in the selected shot image according to a specification operation made on the operation device by the user.

### Claim 7 (depends on 2)

The mobile terminal device according to claim 2, wherein when all of a plurality of face images in the shot image displayed on the display device are each displayed in a representation indicating presence of the modification target region or when, of the plurality of face images in the shot image, a number of face images equal to or more than a predetermined ratio are each displayed in the representation indicating presence of the modification target region, the display controller allows the display device to display a notification indicating that a number of the representations is equal to or more than the predetermined ratio.

### Claim 8 (depends on 1)

The mobile terminal device according to claim 1, wherein,
in addition to an inside camera which is the camera, an outside camera is disposed on a surface of the housing opposite to the particular surface,
the storage device stores the shot images together with respective attendant pieces of camera type information indicating by which of the inside camera and the outside camera the shot image has been taken, and
the display controller reads, among the shot images stored in the storage device, only a shot image the piece of camera type information on which is a piece of information indicating the inside camera, and allows the display device to display the read shot image as a thumbnail index.

### Claim 9 (depends on 1)

The mobile terminal device according to claim 1, wherein when the shot image contains a plurality of persons' face images recognized by the face recognizer, the determiner defines a descending order of facial regions recognized by the face recognizer as a predetermined order, determines, only as for a predetermined number of face images in the predetermined order, whether or not the individual face image is a camera-eye-aligned image, and avoids determining, as for the remaining face images other than the predetermined number, whether or not the individual face image is a camera-eye-aligned image.
