# US 10,771,707 B2 · Information processing device and information processing method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 10,771,707 B2
- Document ID: US10771707B2
- Status: Granted
- Kind: B2
- Application number: 16/336,198
- Filed: 2017-08-21
- Published: 2019-07-25
- Granted: 2020-09-08
- Earliest claimed: 2016-10-17
- Assignees: Sony Corporation
- Applicants: Sony Corporation
- Inventors: Ryouhei Yasuda; Takuro Noda
- Web page: https://patentgrep.com/patent/US10771707B2

## CPC

- G02B 27/0093: {with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking}
- H04N 23/69: Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
- G02B 7/08: adapted to co-operate with a remote control mechanism
- G02B 27/0101: {characterised by optical features (G02B27/0172 takes precedence)}
- G02B 27/0172: {characterised by optical features}
- G03B 13/10: adjusting viewfinders field
- G03B 15/00: Special procedures for taking photographs; Apparatus therefor
- G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics
- H04N 23/60: Control of cameras or camera modules
- H04N 23/61: based on recognised objects
- H04N 23/635: {Region indicators; Field of view indicators}
- H04N 23/695: Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects
- G02B 2027/0138: {comprising image capture systems, e.g. camera}
- G02B 2027/014: {comprising information/image processing systems}
- G02B 2027/0181: {Adaptation to the pilot/driver}
- G03B 2206/00: Systems for exchange of information between different pieces of apparatus, e.g. for exchanging trimming information, for photo finishing
- G03B 2213/025: Sightline detection
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G03B: APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR (optical parts of such apparatus G02B; photosensitive materials or processes for photographic purposes G03C; apparatus for processing exposed photographic materials G03D)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL

## Abstract

[Object] To provide an information processing device, an information processing method, and a program. [Solution] An information processing device including: an acquisition unit configured to acquire attention information regarding an attention target of a user; and a zoom control unit configured to perform zoom control of a camera on the basis of non-zoom control of the camera and the attention information when the non-zoom control is performed on the basis of the attention information.

## Claims

### Claim 1

An information processing device, comprising:
at least one processor configured to:
acquire, from a sensor, attention information that indicates at least one of a gaze direction or a pointing direction of a user toward an attention target of the user;
control a display device to display a field of view of a camera that includes the attention target; and
perform zoom control of the field of view based on non-zoom control of the field of view and the attention information when the non-zoom control is based on the attention information, wherein
the zoom control is further performed based on control of zoom speed of the field of view to suppress separation of the attention target from a center of the field of view while the non-zoom control is performed.

### Claim 2 (depends on 1)

The information processing device according to claim 1, wherein the at least one processor is further configured to control the zoom speed in a manner that the attention target is in the field of view while the zoom control is performed.

### Claim 3 (depends on 1)

The information processing device according to claim 1, wherein the at least one processor is further configured to control the zoom speed in a manner that at least one of an angle change speed ωz of the field of view or a parallel movement speed vz of the field of view in an optical axis direction of the camera satisfies a condition.

### Claim 4 (depends on 3)

The information processing device according to claim 3, wherein the condition is expressed as an expression (A) using an angle of view θs of the camera at a start time of the zoom control, a period t during which the zoom control is performed, a distance h between the attention target and an optical axis of the camera, a distance d between the camera and the attention target in the optical axis direction, a rotation speed ωc of the camera, and a parallel movement speed vh of the camera in a direction perpendicular to the optical axis.
[
Math
.
⁢
16
]
ω
z
≦
θ
s
-
atan
⁡
(
h
-
v
h
⁢
t
d
-
v
z
⁢
t
)
t
+
ω
c
.
(
A
)

### Claim 5 (depends on 3)

The information processing device according to claim 3, wherein the condition is expressed as an expression (B) using a period t during which the zoom control is performed, a distance h between the attention target and an optical axis of the camera, a distance d between the camera and the attention target in the optical axis direction, a rotation speed ωc of the camera, and a parallel movement speed vh of the camera in a direction perpendicular to the optical axis.
[
Math
.
⁢
17
]
ω
c
≦
1
t
⁢
atan
⁡
(
h
-
v
h
⁢
t
d
-
v
z
⁢
t
)
.
(
B
)

### Claim 6 (depends on 1)

The information processing device according to claim 1, wherein the zoom control includes change of a focal length associated with the camera.

### Claim 7 (depends on 1)

The information processing device according to claim 1, wherein the zoom control includes change of a relative distance between the camera and the attention target.

### Claim 8 (depends on 1)

The information processing device according to claim 1, wherein the zoom control is performed substantially simultaneously with the non-zoom control.

### Claim 9 (depends on 1)

The information processing device according to claim 1, wherein the at least one processor is further configured to control the display device to display a guide that indicates that the zoom speed of the field of view is limited in the zoom control.

### Claim 10 (depends on 1)

The information processing device according to claim 1, wherein the non-zoom control is control to move the attention target to a position in the field of view.

### Claim 11 (depends on 1)

The information processing device according to claim 1, wherein
the camera is a real camera in a real space, and
the at least one processor is further configured to control drive of the real camera while the zoom control and the non-zoom control are performed.

### Claim 12

An image processing device, comprising:
at least one processor configured to:
acquire, from a sensor, attention information that indicates at least one of a gaze direction or a pointing direction of a user toward an attention target of the user;
control a display device to display a field of view of a camera that includes the attention target; and
perform zoom control of the field of view based on non-zoom control of the field of view and the attention information when the non-zoom control is based on the attention information, wherein
the camera is a virtual camera that is in a virtual space,
the attention target is a virtual object, and
the non-zoom control includes controlling relative positions of the virtual camera and the virtual object placed in the virtual space.

### Claim 13

An image processing device, comprising:
at least one processor configured to:
acquire, from a sensor, attention information that indicates at least one of a gaze direction or a pointing direction of a user toward an attention target of the user;
control a display device to display a field of view of a camera that includes the attention target; and
perform zoom control of the field of view based on non-zoom control of the field of view and the attention information when the non-zoom control is based on the attention information,
wherein the non-zoom control includes at least one of control of parallel movement of the camera in a direction perpendicular to an optical axis of the camera or control of rotation of the camera.

### Claim 14

An information processing method, comprising:
acquiring, from a sensor, attention information that indicates at least one of a gaze direction or a pointing direction of a user toward an attention target of the user;
controlling a display device to display a field of view of a camera that includes the attention target; and
performing, by a processor, zoom control of the field of view based on non-zoom control of the field of view and the attention information when the non-zoom control is based on the attention information, wherein
the zoom control is further performed based on control of zoom speed of the field of view to suppress separation of the attention target from a center of the field of view while the non-zoom control is performed.

### Claim 15

A non-transitory computer-readable medium having stored thereon computer-readable instructions, which when executed by an image processing device, cause the image processing device to execute operations, the operations comprising:
acquiring, from a sensor, attention information that indicates at least one of a gaze direction or a pointing direction of a user toward an attention target of the user;
controlling a display device to display a field of view of a camera that includes the attention target; and
performing zoom control of the field of view based on non-zoom control of the field of view and the attention information when the non-zoom control is based on the attention information, wherein
the zoom control is further performed based on control of zoom speed of the field of view to suppress separation of the attention target from a center of the field of view while the non-zoom control is performed.