# US 10,372,208 B2 · Power efficient image sensing apparatus, method of operating the same and eye/gaze tracking system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,372,208 B2 - Document ID: US10372208B2 - Status: Granted - Kind: B2 - Application number: 15/806,946 - Filed: 2017-11-08 - Published: 2018-03-08 - Granted: 2019-08-06 - Earliest claimed: 2013-04-29 - Assignees: Tobii AB - Applicants: Tobii AB - Inventors: Marten Skogo; Henrik Jonsson; Mattias O. Karlsson; Mattias Kuldkepp; John Elvesjo; Ingemar Mattias Karlsson - Web page: https://patentgrep.com/patent/US10372208B2 ## CPC - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - G06F 1/325: {Power saving in peripheral device} - G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)} - H04N 13/383: for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes - H04N 23/61: based on recognised objects - H04N 23/651: {for reducing power consumption by affecting camera operations, e.g. sleep mode, hibernation mode or power off of selective parts of the camera} - H04N 23/667: Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes - H04N 25/00: Circuitry of solid-state image sensors [SSIS]; Control thereof - G03B 2213/025: Sightline detection - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G03B: APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR (optical parts of such apparatus G02B; photosensitive materials or processes for photographic purposes G03C; apparatus for processing exposed photographic materials G03D) ## Abstract An image sensing apparatus has a light sensitive area and a control unit. The light sensitive area registers image data in response to an incoming amount of light, and is operable in an active mode wherein image data can be read out there from as well as in a standby mode wherein image data cannot be read out. The control unit produces a control signal setting the light sensitive area to operate in the active mode and the standby mode respectively in a cyclic manner during an operation period, which preferably encompasses multiple data frame read-outs of image data from the light sensitive area. ## Claims ### Claim 1 An image sensing apparatus comprising: a light sensitive area configured to register image data in response to an incoming amount of light, the light sensitive area being operable in an active mode wherein image data can be read out there from and a standby mode wherein image data cannot be read out; a control unit configured to produce a control signal for setting the light sensitive area in the active mode and the standby mode respectively, said control unit configured to produce the control signal such that the light sensitive area transitions from operation in the active mode to the standby mode and back to the active mode in a cyclic manner during an operation period; and an output interface configured to deliver output image data from the light sensitive area, wherein one data frame represents readout of the image data having been registered by the light sensitive area during a capture time, wherein the light sensitive area is configured to feed a sequence of said data frames through the output interface, wherein the control unit is configured to produce the control signal so that the operation period extends over at least one capture time and at least two periods during which the light sensitive area operates in the standby mode, the light sensitive area operating in the active mode during said at least one capture time, and wherein prior to the operation period, the control unit is configured to produce the control signal so that the light sensitive area operates in the active mode during an unbroken period of time exceeding a typical capture time associated with a full readout of image data from the light sensitive area. ### Claim 2 (depends on 1) The image sensing apparatus according to claim 1, wherein the operation period extends over at least two capture times during which the light sensitive area operates in the active mode, said at least two capture times being separated by a respective delay period during which the light sensitive area operates in the standby mode. ### Claim 3 (depends on 1) The image sensing apparatus according to claim 1, wherein the light sensitive area comprises a set of light sensitive elements arranged in a first number of columns and a second number of rows, the light sensitive area is controllable to read out image data to the output interface, which image data have been registered exclusively by at least one subset of the first number of columns and/or the second number of rows, and the control unit is further configured to produce the control signal so that said at least one subset of image data is fed to the output interface when the light sensitive area operates in the active mode. ### Claim 4 (depends on 1) The image sensing apparatus according to claim 1, wherein the light sensitive area comprises a set of light sensitive elements arranged in a first number of columns and a second number of rows, and the light sensitive area is controllable to read out image data to the output interface such that data from two or more light sensitive elements are combined with one another to form a common data unit and a total number of said data units fed to the output interface is less than the first number times the second number. ### Claim 5 An image sensing apparatus comprising: a light sensitive area configured to register image data in response to an incoming amount of light, the light sensitive area being operable in an active mode wherein image data can be read out there from and a standby mode wherein image data cannot be read out; a control unit configured to produce a control signal for setting the light sensitive area in the active mode and the standby mode respectively, said control unit configured to produce the control signal such that the light sensitive area transitions from operation in the active mode to the standby mode and back to the active mode in a cyclic manner during an operation period; and an output interface configured to deliver output image data from the light sensitive area, wherein one data frame represents readout of the image data having been registered by the light sensitive area during a capture time, wherein the light sensitive area is configured to feed a sequence of said data frames through the output interface, wherein the control unit is configured to produce the control signal so that the operation period extends over at least one capture time and at least two periods during which the light sensitive area operates in the standby mode, the light sensitive area operating in the active mode during said at least one capture time, and wherein after the operation period, the control unit is configured to produce the control signal so that the light sensitive area operates in the active mode during an unbroken period of time exceeding a typical capture time associated with a full readout of image data from the light sensitive area. ### Claim 6 (depends on 5) The image sensing apparatus according to claim 5, wherein the operation period extends over at least two capture times during which the light sensitive area operates in the active mode, said at least two capture times being separated by a respective delay period during which the light sensitive area operates in the standby mode. ### Claim 7 (depends on 5) The image sensing apparatus according to claim 5, wherein the light sensitive area comprises a set of light sensitive elements arranged in a first number of columns and a second number of rows, the light sensitive area is controllable to read out image data to the output interface, which image data have been registered exclusively by at least one subset of the first number of columns and/or the second number of rows, and the control unit is further configured to produce the control signal so that said at least one subset of image data is fed to the output interface when the light sensitive area operates in the active mode. ### Claim 8 (depends on 5) The image sensing apparatus according to claim 5, wherein the light sensitive area comprises a set of light sensitive elements arranged in a first number of columns and a second number of rows, and the light sensitive area is controllable to read out image data to the output interface such that data from two or more light sensitive elements are combined with one another to form a common data unit and a total number of said data units fed to the output interface is less than the first number times the second number. ### Claim 9 An eye/gaze tracking system comprising an image sensing apparatus, said image sensing apparatus comprising: a light sensitive area configured to register image data in response to an incoming amount of light, the light sensitive area being operable in an active mode wherein image data can be read out there from and a standby mode wherein image data cannot be read out, wherein the light sensitive area comprises a set of light sensitive elements arranged in a first number of columns and a second number of rows, and wherein the light sensitive area is controllable to read out image data to an output interface, which image data have been registered exclusively by at least one sub-set of the first number of columns and/or the second number of rows; a control unit configured to produce a control signal for setting the light sensitive area in the active mode and the standby mode respectively, said control unit configured to produce the control signal such that the light sensitive area transitions from operation in the active mode to the standby mode and back to the active mode in a cyclic manner during an operation period, wherein the control unit is further configured to produce the control signal so that said at least one subset of image data is fed to the output interface when the light sensitive area operates in the active mode; and an output interface configured to deliver output image data from the light sensitive area, wherein one data frame represents readout of the image data having been registered by the light sensitive area during a capture time, wherein the light sensitive area is configured to feed a sequence of said data frames through the output interface, wherein the control unit is configured to produce the control signal so that the operation period extends over at least one capture time and at least two periods during which the light sensitive area operates in the standby mode, the light sensitive area operating in the active mode during said at least one capture time, wherein prior to the operation period, the control unit is configured to produce the control signal so that the light sensitive area operates in the active mode during an unbroken period of time exceeding a typical capture time associated with a full readout of image data from the light sensitive area, wherein the image sensing apparatus being arranged to capture image data of a subject whose eye and or gaze is tracked, and wherein the control unit is configured to produce the control signal so that said at least one subset of image data represents the image of at least one eye of the subject. ### Claim 10 A method of controlling an image sensing apparatus having a light sensitive area configured to register image data in response to an incoming amount of light, the light sensitive area being operable in an active mode wherein image data can be read out there from and a standby mode wherein image data cannot be read out, and said image sensing apparatus further having an output interface configured to deliver output image data from the light sensitive area; the method comprising: producing a control signal for setting the light sensitive area in the active mode and the standby mode respectively; producing the control signal such that the light sensitive area transitions from operation in the active mode to the standby mode and back to the active mode in a cyclic manner during an operation period; reading out one data frame representing image data having been registered by the light sensitive area during a capture time; feeding out a sequence of said data frames from the light sensitive area through the output interface; and producing the control signal so that the operation period extends over at least one capture time and at least two periods during which the light sensitive area operates in the standby mode, the light sensitive area operating in the active mode during said at least one capture time, wherein, prior to the operation period, said producing comprises producing the control signal so that the light sensitive area operates in the active mode during an unbroken period of time exceeding a typical capture time associated with a full readout of image data from the light sensitive area. ### Claim 11 (depends on 10) The method according to claim 10, comprising producing the control signal so that the operation period extends over at least two capture times during which the light sensitive area operates in the active mode, said at least two capture times being separated by a respective delay period during which the light sensitive area operates in the standby mode. ### Claim 12 (depends on 10) The method according to claim 10, wherein the light sensitive area comprises a set of light sensitive elements arranged in a first number of columns and a second number of rows, and the method further comprises: reading out image data from the light sensitive area to the output interface, which image data have been registered exclusively by at least one subset of the first number of columns and/or the second number of rows; and producing the control signal so that said at least one subset of image data is fed to the output interface when the light sensitive area operates in the active mode. ### Claim 13 (depends on 10) The method according to claim 10, wherein the light sensitive area comprises a set of light sensitive elements arranged in a first number of columns and a second number of rows, and the method further comprises: reading out image data from the light sensitive area to the output interface such that data from two or more light sensitive elements are combined with one another to form a common data unit and a total number of said data units fed to the output interface is less than the first number times the second number. ### Claim 14 A computer program product stored on a non-transitory computer readable medium, comprising software instructions for operation by a computer controller for controlling an image sensing apparatus having a light sensitive area configured to register image data in response to an incoming amount of light, the light sensitive area being operable in an active mode wherein image data can be read out there from and a standby mode wherein image data cannot be read out, and said image sensing apparatus further having an output interface configured to deliver output image data from the light sensitive area, said instructions comprising: first instructions for producing a control signal for setting the light sensitive area in the active mode and the standby mode respectively; second instructions for producing the control signal such that the light sensitive area transitions from operation in the active mode to the standby mode and back to the active mode in a cyclic manner during an operation period; third instructions for reading out one data frame representing image data having been registered by the light sensitive area during a capture time; and fourth instructions for feeding out a sequence of said data frames from the light sensitive area through the output interface, wherein said second instructions includes instructions for producing the control signal so that the operation period extends over at least one capture time and at least two periods during which the light sensitive area operates in the standby mode, the light sensitive area operating in the active mode during said at least one capture time, and wherein said second instructions comprising, prior to the operation period, instructions for producing the control signal so that the light sensitive area operates in the active mode during an unbroken period of time exceeding a typical capture time associated with a full readout of image data from the light sensitive area. ### Claim 15 (depends on 14) The computer program product of claim 14, wherein said second instructions comprises instructions for producing the control signal so that the operation period extends over at least two capture times during which the light sensitive area operates in the active mode, said at least two capture times being separated by a respective delay period during which the light sensitive area operates in the standby mode. ### Claim 16 (depends on 14) The computer program product of claim 14, wherein the light sensitive area comprises a set of light sensitive elements arranged in a first number of columns and a second number of rows, wherein said third instructions comprises instructions for reading out image data from the light sensitive area to the output interface, which image data have been registered exclusively by at least one subset of the first number of columns and/or the second number of rows, and said second instructions comprises instructions for producing the control signal so that said at least one subset of image data is fed to the output interface when the light sensitive area operates in the active mode. ### Claim 17 (depends on 14) The computer program product of claim 14, wherein the light sensitive area comprises a set of light sensitive elements arranged in a first number of columns and a second number of rows, and the method further comprises: second instructions for reading out image data from the light sensitive area to the output interface such that data from two or more light sensitive elements are combined with one another to form a common data unit and a total number of said data units fed to the output interface is less than the first number times the second number. ### Claim 18 An eye/gaze tracking system comprising an image sensing apparatus, said image sensing apparatus comprising: a light sensitive area configured to register image data in response to an incoming amount of light, the light sensitive area being operable in an active mode wherein image data can be read out there from and a standby mode wherein image data cannot be read out, wherein the light sensitive area comprises a set of light sensitive elements arranged in a first number of columns and a second number of rows, and wherein the light sensitive area is controllable to read out image data to an output interface, which image data have been registered exclusively by at least one sub-set of the first number of columns and/or the second number of rows; a control unit configured to produce a control signal for setting the light sensitive area in the active mode and the standby mode respectively, said control unit configured to produce the control signal such that the light sensitive area transitions from operation in the active mode to the standby mode and back to the active mode in a cyclic manner during an operation period, wherein the control unit is further configured to produce the control signal so that said at least one subset of image data is fed to the output interface when the light sensitive area operates in the active mode; and an output interface configured to deliver output image data from the light sensitive area, wherein one data frame represents readout of the image data having been registered by the light sensitive area during a capture time, wherein the light sensitive area is configured to feed a sequence of said data frames through the output interface, wherein the control unit is configured to produce the control signal so that the operation period extends over at least one capture time and at least two periods during which the light sensitive area operates in the standby mode, the light sensitive area operating in the active mode during said at least one capture time, wherein after the operation period, the control unit is configured to produce the control signal so that the light sensitive area operates in the active mode during an unbroken period of time exceeding a typical capture time associated with a full readout of image data from the light sensitive area, wherein the image sensing apparatus being arranged to capture image data of a subject whose eye and or gaze is tracked, and wherein the control unit is configured to produce the control signal so that said at least one subset of image data represents the image of at least one eye of the subject. ### Claim 19 A method of controlling an image sensing apparatus having a light sensitive area configured to register image data in response to an incoming amount of light, the light sensitive area being operable in an active mode wherein image data can be read out there from and a standby mode wherein image data cannot be read out, and said image sensing apparatus further having an output interface configured to deliver output image data from the light sensitive area; the method comprising: producing a control signal for setting the light sensitive area in the active mode and the standby mode respectively; producing the control signal such that the light sensitive area transitions from operation in the active mode to the standby mode and back to the active mode in a cyclic manner during an operation period; reading out one data frame representing image data having been registered by the light sensitive area during a capture time; feeding out a sequence of said data frames from the light sensitive area through the output interface; and producing the control signal so that the operation period extends over at least one capture time and at least two periods during which the light sensitive area operates in the standby mode, the light sensitive area operating in the active mode during said at least one capture time, wherein, after the operation period, said producing comprises producing the control signal so that the light sensitive area operates in the active mode during an unbroken period of time exceeding a typical capture time associated with a full readout of image data from the light sensitive area. ### Claim 20 (depends on 19) The method according to claim 19, comprising producing the control signal so that the operation period extends over at least two capture times during which the light sensitive area operates in the active mode, said at least two capture times being separated by a respective delay period during which the light sensitive area operates in the standby mode. ### Claim 21 (depends on 19) The method according to claim 19, wherein the light sensitive area comprises a set of light sensitive elements arranged in a first number of columns and a second number of rows, and the method further comprises: reading out image data from the light sensitive area to the output interface, which image data have been registered exclusively by at least one subset of the first number of columns and/or the second number of rows; and producing the control signal so that said at least one subset of image data is fed to the output interface when the light sensitive area operates in the active mode. ### Claim 22 (depends on 19) The method according to claim 19, wherein the light sensitive area comprises a set of light sensitive elements arranged in a first number of columns and a second number of rows, and the method further comprises: reading out image data from the light sensitive area to the output interface such that data from two or more light sensitive elements are combined with one another to form a common data unit and a total number of said data units fed to the output interface is less than the first number times the second number. ### Claim 23 A computer program product stored on a non-transitory computer readable medium, comprising software instructions for operation by a computer controller for controlling an image sensing apparatus having a light sensitive area configured to register image data in response to an incoming amount of light, the light sensitive area being operable in an active mode wherein image data can be read out there from and a standby mode wherein image data cannot be read out, and said image sensing apparatus further having an output interface configured to deliver output image data from the light sensitive area, said instructions comprising: first instructions for producing a control signal for setting the light sensitive area in the active mode and the standby mode respectively; second instructions for producing the control signal such that the light sensitive area transitions from operation in the active mode to the standby mode and back to the active mode in a cyclic manner during an operation period; third instructions for reading out one data frame representing image data having been registered by the light sensitive area during a capture time; and fourth instructions for feeding out a sequence of said data frames from the light sensitive area through the output interface, wherein said second instructions includes instructions for producing the control signal so that the operation period extends over at least one capture time and at least two periods during which the light sensitive area operates in the standby mode, the light sensitive area operating in the active mode during said at least one capture time, and wherein said second instructions comprising, after the operation period, instructions for producing the control signal so that the light sensitive area operates in the active mode during an unbroken period of time exceeding a typical capture time associated with a full readout of image data from the light sensitive area. ### Claim 24 (depends on 23) The computer program product of claim 23, wherein said second instructions comprises instructions for producing the control signal so that the operation period extends over at least two capture times during which the light sensitive area operates in the active mode, said at least two capture times being separated by a respective delay period during which the light sensitive area operates in the standby mode. ### Claim 25 (depends on 23) The computer program product of claim 23, wherein the light sensitive area comprises a set of light sensitive elements arranged in a first number of columns and a second number of rows, wherein said third instructions comprises instructions for reading out image data from the light sensitive area to the output interface, which image data have been registered exclusively by at least one subset of the first number of columns and/or the second number of rows, and said second instructions comprises instructions for producing the control signal so that said at least one subset of image data is fed to the output interface when the light sensitive area operates in the active mode. ### Claim 26 (depends on 23) The computer program product of claim 23, wherein the light sensitive area comprises a set of light sensitive elements arranged in a first number of columns and a second number of rows, and the method further comprises: second instructions for reading out image data from the light sensitive area to the output interface such that data from two or more light sensitive elements are combined with one another to form a common data unit and a total number of said data units fed to the output interface is less than the first number times the second number.