# US 10,120,438 B2 · Eye gaze to alter device behavior > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,120,438 B2 - Document ID: US10120438B2 - Status: Granted - Kind: B2 - Application number: 13/115,927 - Filed: 2011-05-25 - Published: 2012-11-29 - Granted: 2018-11-06 - Earliest claimed: 2011-05-25 - Assignees: Sony Interactive Entertainment Inc. - Applicants: Steven Osman; Jeffrey Roger Stafford - Inventors: Steven Osman; Jeffrey Roger Stafford - Web page: https://patentgrep.com/patent/US10120438B2 ## CPC - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - B60K 35/10: Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor - B60K 35/22: Display screens - B60K 35/60: Instruments characterised by their location or relative disposition in or on vehicles (arrangements of lighting devices on dashboards B60Q3/10) - B60K 37/20: Dashboard panels - G06F 1/3231: Monitoring the presence, absence or movement of users - G06F 1/325: {Power saving in peripheral device} - G06F 1/3265: {Power saving in display device} - G06F 1/3287: by switching off individual functional units in the computer system - G06F 3/048: Interaction techniques based on graphical user interfaces [GUI] - G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) - B60K 2360/149: Instrument input by detecting viewing direction not otherwise provided for - H04N 5/76: Television signal recording - Y02D 10/00: Energy efficient computing, e.g. low power processors, power management or thermal management - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - B60K: ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - Y02D: CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE ## Abstract Methods, devices, and computer programs for controlling behavior of an electronic device are presented. A method includes an operation for operating the electronic device in a first mode of operation, and an operation for tracking the gaze of a user interfacing with the electronic device. The electronic device is maintained in a first mode of operation as long as the gaze is directed towards a predetermined target. In another operation, when the gaze is not detected to be directed towards the predetermined target, the electronic device is operated in a second mode of operation, which is different from the first mode of operation. ## Claims ### Claim 1 A method for controlling behavior of an electronic device, the method comprising: operating the electronic device to present on a display a picture-in-picture display, the display having a first area and a second area, wherein a First video is presented in the first area of the display and a second video is presented in the second area of the display for the picture-in-picture display, the first video is associated with a first audio component and the second video is associated with a second audio component, the first audio component is presented: tracking a gaze of a user interfacing with the electronic device; maintaining the presenting of the first video in the first area the second video in the second area, and the first audio component while the gaze is directed to the first area of the display, but away from the second area of the display; detecting that the gaze is directed towards the second area of the display for a predetermined amount of time; switching, in response to the detecting that the gaze is directed towards the second area, a placement of the first video and the second video such that the second video is displayed in the first area and the first video is displayed in the second area; and presenting the second audio component associated with the second video and muting the first audio component associated with the first video in response to the detecting that the gaze is directed toward the second area. ### Claim 2 (depends on 1) The method as recited in claim 1, further including: changing a mode of operation when the gaze of the user is directed away from the display, wherein the changing the mode of operation includes one or more of pausing the first video, recording the first video, raising or lowering a volume of the first video, and turning off the electronic device. ### Claim 3 (depends on 1) The method as recited in claim 1, wherein tracking the gaze of the user further includes: checking a direction of the gaze of the user periodically. ### Claim 4 (depends on 3) The method as recited in claim 3, wherein detecting the gaze further includes: determining that the gaze is directed towards the second area when the gaze is directed towards the second area for a plurality of consecutive periods of checking the direction of the gaze. ### Claim 5 (depends on 1) The method as recited in claim 1, further including: powering off the electronic device after detecting the gaze is away from the display for a predetermined amount of time. ### Claim 6 (depends on 5) The method as recited in claim 5, further including: suspending the electronic device alter detecting the gaze is away from the display for a predetermined amount of time. ### Claim 7 (depends on 1) The method as recited in claim 1, wherein tracking the gaze of the user includes operating a plurality of gaze detecting units. ### Claim 8 An electronic device, comprising: a display with picture-in-picture display for showing a first video in a first area of the display and a second video is presented in a second area of the display for the picture-in-picture display, wherein the first video is associated with a first audio component and the second video is associated with a second audio component; a plurality of gaze detection modules, wherein information from the plurality of gaze detection modules is combined to determine a direction of a gaze of a user interfacing with the display; a processor that detects if the direction of the gaze is directed towards the first area or the second area, wherein the processor switches a placement of the first video and the second video when the processor detects that the gaze is directed towards the second area for a predetermined amount of time, such that the first video is shown in the second area of the display and the second video is shown in the first area of the display; and wherein the first audio component is presented and the second audio component is muted when the first video is shown in the first area of the display and wherein the second audio component is presented and the first audio component is muted when the second video is shown in the first area of the display. ### Claim 9 (depends on 1) The method as recited in claim 1, wherein the switching the placement of the first video and the second video further includes: prompting the user for authorization to switch the placement of the first video and the second video when the gaze is detected to be directed towards the second area of the display; and switching the placement of the first video and the second video based on a response from the user to the prompting. ### Claim 10 (depends on 8) The electronic device of claim 8, wherein the processor is configured to change a mode of operation when the gaze of the user is directed away from the display, wherein the changing the mode of operation includes one or more of pausing the first video, recording the First video, raising or lowering a volume of the first video, and turning off the electronic device. ### Claim 11 (depends on 8) The electronic device of claim 8, wherein the processor is configured to check the direction of the gaze of the user periodically. ### Claim 12 (depends on 11) The electronic device of claim 11, wherein the processor is further configured for determining that the gaze is directed towards the second area when the gaze is directed towards the second area for a plurality of consecutive periods of checking the direction of the gaze. ### Claim 13 (depends on 8) The electronic device of claim 8, wherein the processor is further configured to power off the electronic device after detecting the gaze is away from the display for a predetermined amount of time. ### Claim 14 (depends on 8) The electronic device of claim 8, wherein the processor is further configured to suspend the electronic device after detecting the gaze is away from the display for a predetermined amount of time. ### Claim 15 (depends on 8) The electronic device of claim 8, further comprising: a plurality of gaze detection units for tracking the direction of the gaze.