# US 10,872,261 B2 · Dynamic binning of sensor pixels > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,872,261 B2 - Document ID: US10872261B2 - Status: Granted - Kind: B2 - Application number: 16/208,416 - Filed: 2018-12-03 - Published: 2020-06-04 - Granted: 2020-12-22 - Earliest claimed: 2018-12-03 - Assignees: QUALCOMM Incorporated - Applicants: QUALCOMM Incorporated - Inventors: Soman Ganesh Nikhara; Bapineedu Chowdary Gummadi; Edwin Chongwoo Park; Ravi Shankar Kadambala; Ankita Anil Kumar Choudha - Web page: https://patentgrep.com/patent/US10872261B2 ## CPC - G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI] - G06T 7/11: Region-based segmentation - H04N 25/46: by combining or binning pixels - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract Some dynamic pixel binning methods may involve identifying ROI optical sensor data corresponding to optical sensor pixels in a region of interest (ROI). Some such methods also may involve identifying non-ROI optical sensor data corresponding to optical sensor pixels outside the region of interest and determining a first binning factor for the non-ROI optical sensor data. The first binning factor may be based on a number of pixels in the region of interest and a target output data rate. Some such methods also may involve applying the first binning factor to the non-ROI optical sensor data and outputting optical sensor data, including the binned non-ROI optical sensor data, at the target output data rate. ## Claims ### Claim 1 An optical sensing device, comprising: an array of optical sensor pixels; an interface system comprising at least one interface; and a first control system configured for communication with the array of optical sensor pixels, the first control system being further configured for: receiving optical sensor data from the array of optical sensor pixels; identifying first ROI optical sensor data corresponding to optical sensor pixels in a first region of interest; identifying non-ROI optical sensor data corresponding to optical sensor pixels outside the first region of interest; determining a first binning factor for the non-ROI optical sensor data, wherein determining the first binning factor is based, at least in part, on a number of pixels in the region of interest and a target output data rate; applying the first binning factor to the non-ROI optical sensor data, to produce binned non-ROI optical sensor data; determining whether outputting the first ROI optical sensor data and the binned non-ROI optical sensor data would exceed the target output data rate; and outputting, via the interface system, optical sensor data, including the binned non-ROI optical sensor data, at the target output data rate. ### Claim 2 (depends on 1) The optical sensing device of claim 1, wherein the first control system is further configured for providing, via the interface system, the optical sensor data to a second control system comprising at least one processor, and wherein: identifying the first ROI optical sensor data comprises receiving first ROI information, via the interface system, from the second control system; and determining the first binning factor comprises receiving first binning factor information from the second control system. ### Claim 3 (depends on 2) An optical sensing system that includes the optical sensing device of claim 2 and the second control system. ### Claim 4 (depends on 1) The optical sensing device of claim 1, wherein the first control system comprises at least one processor and wherein: identifying the first ROI optical sensor data comprises determining ROI information, by the first control system, based on the optical sensor data received from the array of optical sensor pixels; and determining the first binning factor comprises determining first binning factor information by the first control system based, at least in part, on the number of pixels in the region of interest and the target output data rate. ### Claim 5 (depends on 1) The optical sensing device of claim 1, wherein the first control system determines that outputting first ROI optical sensor data and the binned non-ROI optical sensor data would exceed the target output data rate and wherein the first control system is further configured for: determining a second binning factor for the first ROI optical sensor data, wherein determining the second binning factor is based, at least in part, on the binned non-ROI optical sensor data and the target output data rate; applying the second binning factor to the first ROI optical sensor data to produce binned first ROI optical sensor data; and outputting the binned first ROI optical sensor data and the binned non-ROI optical sensor data. ### Claim 6 (depends on 1) The optical sensing device of claim 1, wherein the first control system is further configured to apply a larger binning factor to the non-ROI optical sensor data prior to determining whether outputting non-binned ROI optical sensor data and the binned first non-ROI optical sensor data would exceed the target output data rate, the larger binning factor being larger than the first binning factor. ### Claim 7 (depends on 1) The optical sensing device of claim 1, wherein the first control system is further configured for: identifying second ROI optical sensor data corresponding to optical sensor pixels in a second region of interest; determining a first ROI binning factor for the first ROI optical sensor data; determining a second ROI binning factor for the second ROI optical sensor data; applying the first ROI binning factor to the first ROI optical sensor data; and applying the second ROI binning factor to the second ROI optical sensor data. ### Claim 8 (depends on 1) The optical sensing device of claim 1, wherein applying the first binning factor to the non-ROI optical sensor data comprises determining an average value of optical sensor data from two or more optical sensor pixels. ### Claim 9 (depends on 1) The optical sensing device of claim 1, wherein the target output data rate corresponds to a maximum number of optical sensor pixels in the region of interest. ### Claim 10 An optical sensing method, comprising: receiving, by a first control system, optical sensor data from an array of optical sensor pixels; identifying, by the first control system, ROI optical sensor data corresponding to optical sensor pixels in a region of interest; identifying, by the first control system, non-ROI optical sensor data corresponding to optical sensor pixels outside the region of interest; determining, by the first control system, a first binning factor for the non-ROI optical sensor data, wherein determining the first binning factor is based, at least in part, on a number of pixels in the region of interest and a target output data rate; applying, by the first control system, the first binning factor to the non-ROI optical sensor data, to produce binned non-ROI optical sensor data; determining whether outputting the first ROI optical sensor data and the binned non-ROI optical sensor data would exceed the target output data rate; and outputting, via an interface system, optical sensor data, including the binned non-ROI optical sensor data, at the target output data rate. ### Claim 11 (depends on 10) The method of claim 10, further comprising providing, via the interface system, the optical sensor data to a second control system comprising at least one processor, wherein: identifying the ROI optical sensor data comprises receiving ROI information, via the interface system, from the second control system; and determining the first binning factor comprises receiving first binning factor information from the second control system. ### Claim 12 (depends on 10) The method of claim 10, wherein the first control system comprises at least one processor and wherein: identifying the ROI optical sensor data comprises determining ROI information, by the first control system, based on the optical sensor data received from the array of optical sensor pixels; and determining the first binning factor comprises determining first binning factor information by the first control system based, at least in part, on the number of pixels in the region of interest and the target output data rate. ### Claim 13 (depends on 10) The method of claim 10, wherein it is determined that outputting non-binned ROI optical sensor data and the binned non-ROI optical sensor data would exceed the target output data rate and wherein the method further comprises: determining a second binning factor for the ROI optical sensor data, wherein determining the second binning factor is based, at least in part, on the binned non-ROI optical sensor data and the target output data rate; applying the second binning factor to the ROI optical sensor data to produce binned ROI optical sensor data; and outputting the binned ROI optical sensor data and the binned non-ROI optical sensor data. ### Claim 14 (depends on 10) The method of claim 10, further comprising applying a maximum binning factor to the non-ROI optical sensor data prior to determining whether outputting non-binned ROI optical sensor data and the binned non-ROI optical sensor data would exceed the target output data rate. ### Claim 15 (depends on 10) The method of claim 10, further comprising applying a fractional first binning factor to the non-ROI optical sensor data via an extrapolation method. ### Claim 16 (depends on 10) The method of claim 10, wherein applying the first binning factor to the non-ROI optical sensor data comprises determining an average value of optical sensor data from two or more optical sensor pixels. ### Claim 17 (depends on 10) The method of claim 10, wherein the target output data rate corresponds to a maximum number of optical sensor pixels in the region of interest. ### Claim 18 One or more non-transitory media having software stored thereon, the software including instructions for controlling one or more devices to perform an optical sensing method, the method comprising: receiving, by a first control system, optical sensor data from an array of optical sensor pixels; identifying, by the first control system, ROI optical sensor data corresponding to optical sensor pixels in a region of interest; identifying, by the first control system, non-ROI optical sensor data corresponding to optical sensor pixels outside the region of interest; determining, by the first control system, a first binning factor for the non-ROI optical sensor data, wherein determining the first binning factor is based, at least in part, on a number of pixels in the region of interest and a target output data rate; applying, by the first control system, the first binning factor to the non-ROI optical sensor data, to produce binned non-ROI optical sensor data; and determining whether outputting the first ROI optical sensor data and the binned non-ROI optical sensor data would exceed the target output data rate; outputting, via an interface system, optical sensor data, including the binned non-ROI optical sensor data, at the target output data rate. ### Claim 19 (depends on 18) The one or more non-transitory media of claim 18, wherein the method further comprises providing, via the interface system, the optical sensor data to a second control system comprising at least one processor, wherein: identifying the ROI optical sensor data comprises receiving ROI information, via the interface system, from the second control system; and determining the first binning factor comprises receiving first binning factor information from the second control system. ### Claim 20 (depends on 18) The one or more non-transitory media of claim 18, wherein the first control system comprises at least one processor and wherein: identifying the ROI optical sensor data comprises determining ROI information, by the first control system, based on the optical sensor data received from the array of optical sensor pixels; and determining the first binning factor comprises determining first binning factor information by the first control system based, at least in part, on the number of pixels in the region of interest and the target output data rate. ### Claim 21 (depends on 18) The one or more non-transitory media of claim 18, wherein it is determined that outputting non-binned ROI optical sensor data and the binned non-ROI optical sensor data would exceed the target output data rate and wherein the method further comprises: determining a second binning factor for the ROI optical sensor data, wherein determining the second binning factor is based, at least in part, on the binned non-ROI optical sensor data and the target output data rate; applying the second binning factor to the ROI optical sensor data to produce binned ROI optical sensor data; and outputting the binned ROI optical sensor data and the binned non-ROI optical sensor data. ### Claim 22 (depends on 18) The one or more non-transitory media of claim 18, wherein the method further comprises applying a maximum binning factor to the non-ROI optical sensor data prior to determining whether outputting non-binned ROI optical sensor data and the binned non-ROI optical sensor data would exceed the target output data rate. ### Claim 23 (depends on 18) The one or more non-transitory media of claim 18, wherein the method further comprises applying a fractional first binning factor to the non-ROI optical sensor data via an extrapolation method. ### Claim 24 (depends on 18) The one or more non-transitory media of claim 18, wherein applying the first binning factor to the non-ROI optical sensor data comprises determining an average value of optical sensor data from two or more optical sensor pixels. ### Claim 25 (depends on 18) The one or more non-transitory media of claim 18, wherein the target output data rate corresponds to a maximum number of optical sensor pixels in the region of interest. ### Claim 26 An optical sensing device, comprising: an array of optical sensor pixels; an interface system comprising at least one interface; and a first control means for communication with the array of optical sensor pixels, the first control means including means for: receiving optical sensor data from the array of optical sensor pixels; identifying ROI optical sensor data corresponding to optical sensor pixels in a region of interest; identifying non-ROI optical sensor data corresponding to optical sensor pixels outside the region of interest; determining a first binning factor for the non-ROI optical sensor data, wherein determining the first binning factor is based, at least in part, on a number of pixels in the region of interest and a target output data rate; applying the first binning factor to the non-ROI optical sensor data, to produce binned non-ROI optical sensor data; determining whether outputting the first ROI optical sensor data and the binned non-ROI optical sensor data would exceed the target output data rate; and outputting, via the interface system, optical sensor data, including the binned non-ROI optical sensor data, at the target output data rate. ### Claim 27 (depends on 26) An optical sensing system that includes the optical sensing device of claim 26 and also comprises second control means, wherein the first control means includes means for providing, via the interface system, the optical sensor data to the second control means and wherein: identifying the ROI optical sensor data comprises receiving ROI information, via the interface system, from the second control means; and determining the first binning factor comprises receiving first binning factor information from the second control means.