# US 10,120,441 B2 · Controlling display content based on a line of sight of a user > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,120,441 B2 - Document ID: US10120441B2 - Status: Granted - Kind: B2 - Application number: 14/917,244 - Filed: 2014-06-30 - Published: 2016-07-07 - Granted: 2018-11-06 - Earliest claimed: 2013-09-13 - Assignees: Sony Corporation - Applicants: Sony Corporation - Inventors: Maki Imoto; Takuro Noda; Ryouhei Yasuda - Web page: https://patentgrep.com/patent/US10120441B2 ## CPC - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - G06F 3/012: {Head tracking input arrangements} - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)} - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract There is provided an information processing apparatus including an image acquisition unit configured to acquire a captured image of users, a determination unit configured to determine an operator from among the users included in the acquired captured image, and a processing unit configured to conduct a process based on information about user line of sight corresponding to the determined operator. ## Claims ### Claim 1 An information processing apparatus, comprising: a display screen; and one or more processors configured to: acquire a captured image that includes a plurality of users; detect a plurality of face regions that correspond to the plurality of users in the captured image; determine an operator from the plurality of users included in the captured image, based on a position of a line of sight of a first user of the plurality of users in a specific region on the display screen and based on a size of each of the plurality of face regions; and control a first process based on first information associated with the line of sight corresponding to the determined operator. ### Claim 2 (depends on 1) The information processing apparatus according to claim 1, wherein the one or more processors are further configured to determine the first user of the plurality of users as the operator based on a first face region of the plurality of face regions, wherein the first face region corresponds to the first user that has a largest face region size among the plurality of face regions. ### Claim 3 (depends on 2) The information processing apparatus according to claim 2, wherein based on each of a set of face regions of the plurality of face regions, that have a face region size that is largest among the plurality of face regions, the one or more processors are further configured to determine the first face region of the set of face regions as the operator, wherein the first face region is detected prior to a second face region of the set of face regions. ### Claim 4 (depends on 3) The information processing apparatus according to claim 3, wherein the one or more processors are further configured to determine a configured number of users of the plurality of users as the operator, based on the set of face regions that have the face region size that is largest among the plurality of face regions. ### Claim 5 (depends on 2) The information processing apparatus according to claim 2, wherein the one or more processors are further configured to determine the first user is a user other than the operator based on a first difference value that is equal to or greater than a first threshold value, the first difference value indicates a difference of a first face region size corresponding to the first user and a second face region size corresponding to a second user other than the first user. ### Claim 6 (depends on 1) The information processing apparatus according to claim 1, wherein the one or more processors are further configured to determine the operator based on a corresponding distance of each face region of the plurality of face regions from the display screen. ### Claim 7 (depends on 6) The information processing apparatus according to claim 6, wherein the one or more processors are further configured to determine the first user of the plurality of users as the operator based on a first face region of the first user that has a shortest distance among the corresponding distance of the each face region of the plurality of face regions from the display screen. ### Claim 8 (depends on 7) The information processing apparatus according to claim 7, wherein based on each of a set of distances having the shortest distance among the corresponding distance of the each face region of the plurality of face regions, the one or more processors are further configured to determine the first user as the operator, wherein the first user that corresponds to the set of distances is detected prior to a second user that corresponds to the set of distances. ### Claim 9 (depends on 8) The information processing apparatus according to claim 8, wherein the one or more processors are further configured to determine a configured number of users of the plurality of users as the operator, based on the set of distances that have the shortest distance. ### Claim 10 (depends on 7) The information processing apparatus according to claim 7, wherein the one or more processors are further configured to determine the first user is a user other than the operator based on a second difference value that is equal to or greater than a second threshold value, the second difference value indicates a difference between a first distance from the display screen and a second distance from the display screen, wherein the first distance corresponds to the first face region of the first user and the second distance corresponds to a second face region of a second user of the plurality of users. ### Claim 11 (depends on 1) The information processing apparatus according to claim 1, wherein the one or more processors are further configured to determine the operator based on a specific gesture detected from the captured image. ### Claim 12 (depends on 1) The information processing apparatus according to claim 1, wherein the one or more processors are further configured to determine the operator based on a rank associated with each of the plurality of users. ### Claim 13 (depends on 1) The information processing apparatus according to claim 1, wherein the one or more processors are further configured to: determine an operation level for the first user of the plurality of users, wherein the first user is determined as the operator; and dynamically change available processes based on the determined operation level. ### Claim 14 (depends on 1) The information processing apparatus according to claim 1, wherein based on a determination that a known speech is spoken by a second user of the plurality of users, the one or more processors are further configured to control a second process based on second information about a line of sight corresponding to the second user. ### Claim 15 (depends on 1) The information processing apparatus according to claim 1, wherein based on an unavailability of the first information, the one or more processors are further configured to determine a new operator from among the plurality of users. ### Claim 16 (depends on 1) The information processing apparatus according to claim 1, wherein based on an unavailability of the first information, the one or more processors are further configured to determine the operator based on a detection of a head among the plurality of users. ### Claim 17 (depends on 1) The information processing apparatus according to claim 1, wherein based on a detection of a known combination of gestures by the first user determined as the operator and a second user other than the first user, the one or more processors are further configured to change the operator from the first user to the second user. ### Claim 18 (depends on 1) The information processing apparatus according to claim 1, wherein the one or more processors are further configured to determine the first user of the plurality of users as the operator based on the position of the line of sight of the first user that is included in the specific region on the display screen, and wherein the first information associated with the line of sight of the first user indicates that the position of the line of sight of the first user is included in the specific region on the display screen. ### Claim 19 An information processing method, comprising: in an information processing apparatus: acquiring a captured image that includes a plurality of users; detecting a plurality of face regions that correspond to the plurality of users in the captured image; determining an operator from the plurality of users included in the captured image, based on a position of a line of sight of a user of the plurality of users in a specific region on a display screen and based on a size of the plurality of face regions of the plurality of users; and controlling a process based on information associated with the line of sight corresponding to the determined operator.