# US 2021/0243348 A1 · Image sensor module image compression method thereof and electronic device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0243348 A1 - Document ID: US20210243348A1 - Status: Published application - Kind: A1 - Application number: 17/038,337 - Filed: 2020-09-30 - Published: 2021-08-05 - Earliest claimed: 2020-02-05 - Assignees: Samsung Electronics Co., Ltd. - Applicants: Samsung Electronics Co., Ltd. - Inventors: Eundoo Heo - Web page: https://patentgrep.com/patent/US20210243348A1 ## CPC - H04N 19/70: characterised by syntax aspects related to video coding, e.g. related to compression standards - H04N 23/73: by influencing the exposure time - G06T 3/40: Scaling of whole images or parts thereof, e.g. expanding or contracting - G06T 5/50: using two or more images, e.g. averaging or subtraction - G06T 5/92: based on global image properties - G06T 9/00: Image coding (bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00) - H04N 19/186: the unit being a colour or a chrominance component - H04N 23/71: Circuitry for evaluating the brightness variation - H04N 23/741: by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors - H04N 23/743: Bracketing, i.e. taking a series of images with varying exposure conditions - G06T 2207/20224: Image subtraction - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract An image sensor module includes an image sensor, the image sensor module being configured such that the image sensor outputs a plurality of images and the plurality of images having a plurality of exposure times different from each other, a reference image selector circuit outputting reference image selection information for selecting a reference image, from among the plurality of images, based on a reference brightness value, and an image compressor circuit outputting a compression image data including at least one packing data and the reference image, based on the reference image selection information. The at least one packing data respectively corresponds to at least one remaining image other than the reference image, from among the plurality of images, and is generated based on a ratio of an exposure time of a corresponding image to an exposure time of the reference image. ## Claims ### Claim 1 An image sensor module comprising: an image sensor, wherein the image sensor module is configured such that the image sensor outputs a plurality of images, and wherein the plurality of images have a plurality of exposure times different from each other; a reference image selector circuit configured to output reference image selection information for selecting a reference image, from among the plurality of images, based on a reference brightness value; and an image compressor circuit configured to output a compression image data including at least one packing data and the reference image, based on the reference image selection information, wherein the at least one packing data respectively corresponds to at least one remaining image other than the reference image, from among the plurality of images, and is generated based on a ratio of an exposure time of a corresponding image to an exposure time of the reference image. ### Claim 2 (depends on 1) The image sensor module of claim 1, wherein the reference image selector circuit is configured to: receive the reference brightness value; and output the reference image selection information based on the reference brightness value, and wherein the reference image selector circuit is configured to select an image, among from the plurality of images, of a shorter exposure time as the reference image using the reference image selection information generated based on a higher reference brightness value. ### Claim 3 (depends on 1) The image sensor module of claim 1, wherein the reference image selector circuit is configured to: determine a brightness value range to which the reference brightness value belongs, from among a plurality of brightness value ranges, based on a table comprising image lists corresponding to the plurality of brightness value ranges; and output the reference image selection information to select an image, among the plurality of images, corresponding to the determined brightness value range as the reference image. ### Claim 4 (depends on 1) The image sensor module of claim 1, wherein the at least one remaining image comprises a first image of the plurality of images, wherein the image compressor circuit is configured to generate first packing data corresponding to the first image by using a first ratio, and wherein the first ratio indicates a ratio of an exposure time of the first image to an exposure time of the reference image. ### Claim 5 (depends on 4) The image sensor module of claim 4, wherein the image compressor circuit is configured to: obtain a first correction reference image by multiplying the reference image by the first ratio; subtract, pixel-by-pixel, a brightness value of the first image from a brightness value of the first correction reference image to generate first raw disparity data; and generate the first packing data corresponding to the first image based on the first raw disparity data. ### Claim 6 (depends on 5) The image sensor module of claim 5, wherein the first raw disparity data include a plurality of element values arranged in a matrix form, wherein the first packing data comprises a plurality of saturation bits, and wherein each of the plurality of saturation bits indicates whether a corresponding element value of the first raw disparity data having the matrix form exceeds a threshold value. ### Claim 7 (depends on 5) The image sensor module of claim 5, wherein the first raw disparity data include a plurality of element values arranged in a matrix form, wherein the first packing data comprises a plurality of sign bits, and wherein each of the plurality of sign bits indicates whether a corresponding element value of the first raw disparity data having the matrix form is below zero. ### Claim 8 (depends on 7) The image sensor module of claim 7, wherein the first packing data further comprises a plurality of absolute values, and wherein each of the plurality of absolute values indicates an absolute value of a corresponding element value of the first raw disparity data. ### Claim 9 (depends on 6) The image sensor module of claim 6, wherein the first packing data comprises a disparity bit indicating a bit number, in a binary form, allocated to represent a value of each of the plurality of element values of the first raw disparity data, and wherein a value of the disparity bit is equal to the threshold value. ### Claim 10 An image compression method of an image sensor module, the method comprising: obtaining a plurality of images corresponding to a plurality of exposure times; selecting a reference image based on a reference brightness value from among the plurality of images; generating at least one packing data, wherein the at least one packing data respectively corresponds to at least one remaining image other than the reference image from among the plurality of images, and wherein each of the at least one packing data comprises a disparity indicating a difference between a correction reference image, which corresponds to the reference image corrected based on an exposure time ratio of an exposure time of a corresponding image to an exposure time of the reference image, and the corresponding image; and outputting the reference image and the at least one packing data as compressed image data. ### Claim 11 (depends on 10) The method of claim 10, wherein the selecting of the reference image comprises selecting an image, from among the plurality of images, corresponding to a shorter exposure time, from among the plurality of images, as the reference brightness value increases. ### Claim 12 (depends on 10) The method of claim 10, wherein the reference brightness value is provided from the outside of the image sensor module or obtained by the image sensor module based on the plurality of images. ### Claim 13 (depends on 10) The method of claim 10, wherein the at least one remaining image comprises a first image of the plurality of images, wherein the generating of the at least one packing data comprises generating first packing data corresponding to the first image, by using a first ratio, wherein the first ratio indicates a ratio of an exposure time of the first image to an exposure time of the reference image. ### Claim 14 (depends on 13) The method of claim 13, wherein the generating of the first packing data comprises: generating a first correction reference image by multiplying the reference image by the first ratio; generating first raw disparity data by subtracting, pixel-by-pixel, a brightness value of the first image from a brightness value of the first correction reference image; and generating the first packing data based on the first raw disparity data. ### Claim 15 (depends on 14) The method of claim 14, wherein the first raw disparity data include a plurality of element values arranged in a matrix form, and wherein the generating of the first packing data based on the first raw disparity data comprises: generating a plurality of saturation bits, each of the plurality of saturation bits indicating whether a corresponding element value of the first raw disparity data in the matrix form exceeds a threshold value; generating a plurality of sign bits, each of the plurality of sign bits indicating whether a corresponding element value of the first raw disparity data in the matrix form is below zero; and generating the first packing data using the plurality of saturation bits, the plurality of sign bits, and an absolute value of each of the plurality of element values of the first raw disparity data. ### Claim 16 An electronic device comprising: an image sensor module configured to: generate a plurality of images corresponding to a plurality of exposure times; and output compression image data including at least one packing data and a reference image selected from among the plurality of images, wherein the at least one packing data respectively corresponds to at least one remaining image other than the reference image; and a processor configured to: reconstruct the at least one remaining image from the compression image data, based on information about the reference image, the at least one packing data, and the plurality of exposure times; and generate a high-dynamic-range (HDR) image based on the reference image and the at least one remaining image that is reconstructed. ### Claim 17 (depends on 16) The electronic device of claim 16, wherein the image sensor module comprises: an image sensor, wherein the image sensor module is configured such that the image sensor generates the plurality of images; a reference image selector circuit configured to output reference image selection information for selecting the reference image, from among the plurality of images, based on a reference brightness value; and an image compressor circuit configured to output a compression image data including the reference image and the at least one packing data, based on the reference image selection information. ### Claim 18 (depends on 17) The electronic device of claim 17, wherein the reference image selector circuit is configured to: receive the reference brightness value; and output the reference image selection information, and wherein the reference image selector circuit is configured to select an image, among from the plurality of images, of a shorter exposure time as the reference image using the reference image selection information generated based on a higher reference brightness value. ### Claim 19 (depends on 17) The electronic device of claim 17, wherein the at least one remaining image comprises a first image of the plurality of images, wherein the image compressor circuit is configured to: obtain a first correction reference image by multiplying the reference image by a first ratio; subtract, pixel-by-pixel, a brightness value of the first image from a brightness value of the first correction reference image to generate a plurality of first raw disparity data; and generate first packing data corresponding to the first image based on the plurality of first raw disparity data, and wherein the first ratio indicates a ratio of an exposure time of the first image to an exposure time of the reference image. ### Claim 20 (depends on 19) The electronic device of claim 19, wherein the image compressor circuit generates the first packing data using a plurality of saturation bits, a plurality of sign bits, and a plurality of first disparity data, wherein each of the plurality of saturation bits indicates whether a corresponding value of the plurality of first raw disparity data exceeds a threshold value, wherein each of the plurality of sign bits indicates whether a corresponding value of the plurality of first raw disparity data is below zero, and wherein each of the plurality of first disparity data indicates an absolute value of a corresponding value of the plurality of first raw disparity data.