# US 10,313,648 B2 · Image capturing apparatus and image capturing method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,313,648 B2 - Document ID: US10313648B2 - Status: Granted - Kind: B2 - Application number: 15/855,060 - Filed: 2017-12-27 - Published: 2018-05-03 - Granted: 2019-06-04 - Earliest claimed: 2006-10-30 - Assignees: Sony Corporation - Applicants: Sony Corporation - Inventors: Ryota Kosakai; Katsutoshi Aiki; Nobuyuki Sato; Hiroki Nagahama; Masatoshi Sase; Yutaka Yoneda - Web page: https://patentgrep.com/patent/US10313648B2 ## CPC - H04N 23/88: for colour balance, e.g. white-balance circuits or colour temperature control - G11B 27/005: {Reproducing at a different information rate from the information rate of recording (for television signals H04N5/783)} - G11B 27/031: Electronic editing of digitised analogue information signals, e.g. audio or video signals - G11B 31/006: {with video camera or receiver} - H04N 5/77: between a recording apparatus and a television camera - H04N 5/772: {the recording apparatus and the television camera being placed in the same enclosure} - H04N 5/783: Adaptations for reproducing at a rate different from the recording rate - H04N 5/91: Television signal processing therefor - H04N 7/0127: {by changing the field or frame frequency of the incoming video signal, e.g. frame rate converter} - H04N 9/7904: {using intermediate digital signal processing} - H04N 9/80: Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback - H04N 9/87: Regeneration of colour television signals (H04N9/80 takes precedence) - H04N 23/60: Control of cameras or camera modules - H04N 23/63: by using electronic viewfinders - H04N 23/667: Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes - H04N 23/83: specially adapted for colour signals - H04N 23/843: {Demosaicing, e.g. interpolating colour pixel values} - H04N 23/951: by using two or more images to influence resolution, frame rate or aspect ratio - H04N 25/134: {based on three different wavelength filter elements} - H04N 7/0105: {using a storage device with different write and read speed} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G11B: INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER (recording measured values in a way that does not require playback through a transducer G01D9/00; recording or playback apparatus using mechanically marked tape, e.g. punched paper tape, or using unit records, e.g. punched or magnetically marked cards G06K; transferring data from one type of record carrier to another G06K1/18; circuits for coupling output of reproducer to radio receiver H04B1/20; gramophone pick-ups or like acoustic electromechanical transducers or circuits therefor H04R) ## Abstract In a high speed image capturing state, a camera signal processing circuit is not needed to perform a signal process at a high screen rate, but at a regular screen rate. In the high speed image capturing mode, raw data of 240 fps received from an image sensor 101 are recorded on a recording device 111 through a conversion processing section 201 and a recording device controlling circuit 210. Raw data that have been decimated and size-converted are supplied to a camera signal processing circuit 203 through a pre-processing circuit 202 and an image being captured is displayed on a display section 112 with a signal for which a camera process has been performed. In a reproducing state, raw data are read from the recording device 111 at a low screen rate according to a display performance of the display section 112 and the raw data that have been read are processed are processed by the pre-processing circuit 202 and the camera signal processing circuit 203 and a reproduced image is displayed by the display section 112. ## Claims ### Claim 1 An imaging apparatus, comprising: an image sensor that includes a color filter array and is configured to output at a first frame rate a first number of first frames of image data that have not yet been subjected to white balance processing, gamma correction processing, and YC conversion processing; at least one processor; and at least one first storage medium encoded with executable instructions that, when executed by the at least one processor, cause the at least one processor to control circuitry of the apparatus so as to cause the circuitry to: in a recording process, compress the first frames of image data, without previously performing white balance processing, gamma correction processing, and YC conversion processing thereon, to generate compressed image data; store the compressed image data, without performing white balance processing, gamma correction processing, and YC conversion processing thereon, on a second storage medium while the second storage medium is attached to the imaging apparatus, the second storage medium being configured to be detachable from the imaging apparatus as to allow the image data to be processed by a device separate from the imaging apparatus when the second storage medium is detached from the imaging apparatus; and generate a first signal representing a first white-balanced, gamma-corrected, and YC converted image sequence that is to be displayed on a display as a camera-through image by processing the first frames of image data to generate a second number of second frames of image data that is lower than the first number of first frames of image data, and by performing at least one of white balance processing, gamma correction processing, or YC conversion processing on the second frames of image data at a second frame rate which is lower than the first frame rate. ### Claim 2 (depends on 1) The imaging apparatus of claim 1, wherein the at least one first storage medium is further encoded with executable instructions that, when executed by the at least one processor, cause the at least one processor to control the circuitry of the apparatus so as to cause the circuitry to: in a reproduction process, read the compressed image data from the second storage medium; decompress the compressed image data to produce decompressed frames of image data; and generate a second signal representing a second white-balanced, gamma-corrected, and YC converted image sequence to be displayed on the display as a slow motion reproduced image by processing the decompressed image data. ### Claim 3 (depends on 2) The imaging apparatus of claim 2, wherein the at least one first storage medium is further encoded with executable instructions that, when executed by the at least one processor, cause the at least one processor to control the circuitry of the apparatus so as to cause the circuitry to: in the reproduction process, generate the second signal by processing the decompressed image data at a third frame rate which is lower than the first frame rate. ### Claim 4 (depends on 2) The imaging apparatus of claim 2, wherein: the image sensor is further configured to output the first number of first frames of image data throughout a first time period; and the at least one first storage medium is further encoded with executable instructions that, when executed by the at least one processor, cause the at least one processor to control the circuitry of the apparatus so as to cause the circuitry to, in the reproduction process, generate the second signal so that the second white-balanced, gamma-corrected, and YC converted image sequence is displayed on the display throughout a second time period, which is longer than the first time period. ### Claim 5 (depends on 1) The imaging apparatus of claim 1, wherein the third frame rate is the same as the second frame rate. ### Claim 6 (depends on 1) The imaging apparatus of claim 1, wherein the at least one first storage medium is further encoded with executable instructions that, when executed by the at least one processor, cause the at least one processor to control the circuitry of the apparatus so as to cause the circuitry to: in the recording process, process the first and second frames of image data such that a ratio of the first number of frames to the second number of frames is the same as a ratio of the first frame rate to the second frame rate. ### Claim 7 (depends on 6) The imaging apparatus of claim 6, wherein the at least one first storage medium is further encoded with executable instructions that, when executed by the at least one processor, cause the at least one processor to control the circuitry of the apparatus so as to cause the circuitry to: in the recording process, generate the second number of second frames of image data by decimating the first number of first frames of image data. ### Claim 8 (depends on 1) The imaging apparatus of claim 1, wherein the at least one first storage medium is further encoded with executable instructions that, when executed by the at least one processor, cause the at least one processor to control the circuitry of the apparatus so as to cause the circuitry to: in the recording process, generate the first signal such that the first white-balanced, gamma-corrected, and YC converted image sequence corresponds to at least some of the image data that is currently being captured by the image sensor. ### Claim 9 (depends on 1) The imaging apparatus of claim 1, wherein the at least one first storage medium is further encoded with executable instructions that, when executed by the at least one processor, cause the at least one processor to control the circuitry of the apparatus so as to cause the circuitry to: in the recording process, convert the first number of first frames of image data to the second number of second frames of image data by decimating the first number of first frames of image data. ### Claim 10 (depends on 9) The imaging apparatus of claim 9, wherein the at least one first storage medium is further encoded with executable instructions that, when executed by the at least one processor, cause the at least one processor to control the circuitry of the apparatus so as to cause the circuitry to: in the recording process, convert the first number of first frames of image data to the second number of second frames of image data by performing a size adjustment processing on at least some frames of image data. ### Claim 11 (depends on 1) The imaging apparatus of claim 1, wherein the image sensor is configured to be selectively switched between a high-speed image capturing mode in which the image sensor outputs the first frames of image data at the first frame rate, and a lower-speed image capturing mode in which the image sensor outputs the first frames of image data at the second frame rate. ### Claim 12 (depends on 1) The imaging apparatus of claim 1, wherein the second frame rate is one fourth of the first frame rate. ### Claim 13 (depends on 1) The imaging apparatus of claim 1, wherein the at least one first storage medium is further encoded with executable instructions that, when executed by the at least one processor, cause the at least one processor to control the circuitry of the apparatus so as to cause the circuitry to: in the recording process, generate the first signal by performing a simultaneous forming process on at least some frames of image data. ### Claim 14 (depends on 13) The imaging apparatus of claim 13, Wherein the at least one first storage medium is further encoded with executable instructions that, when executed by the at least one processor, cause the at least one processor to control the circuitry of the apparatus so as to cause the circuitry to: in the recording process, generate the first signal by performing each of white balance processing, gamma correction processing, and YC conversion processing on the second frames of image data. ### Claim 15 (depends on 1) The imaging apparatus of claim 1, wherein the display comprises a liquid crystal display and the at least one first storage medium is further encoded with executable instructions that, when executed by the at least one processor, cause the at least one processor to control the circuitry of the apparatus so as to cause the circuitry to: in the recording process, generate the first signal by performing display processing on the second frames of image data, after performing the at least one of white balance processing, gamma correction processing, and YC conversion processing thereon, and providing the first signal to the liquid crystal display so as to cause the liquid crystal display to display the first white-balanced, gamma-corrected, and YC converted image sequence. ### Claim 16 (depends on 1) The imaging apparatus of claim 1, wherein the image sensor is further configured to generate the first frames of image data at a frame rate of approximately 240 frames per second. ### Claim 17 (depends on 1) The imaging apparatus of claim 1, wherein the at least one first storage medium is further encoded with executable instructions that, when executed by the at least one processor, cause the at least one processor to control the circuitry of the apparatus so as to cause the circuitry to: in the recording process, compress the first frames of image data, without previously performing simultaneous forming processing thereon, to generate the compressed image data stored on the second storage medium. ### Claim 18 (depends on 1) The imaging apparatus of claim 1, wherein the at least one first storage medium is further encoded with executable instructions that, when executed by the at least one processor, cause the at least one processor to control the circuitry of the apparatus so as to cause the circuitry to: in the recording process, compress the first frames of image data, without previously performing white balance processing thereon, to generate the compressed image data stored on the second storage medium. ### Claim 19 (depends on 1) The imaging apparatus of claim 1, wherein the at least one first storage medium is further encoded with executable instructions that, when executed by the at least one processor, cause the at least one processor to control the circuitry of the apparatus so as to cause the circuitry to: in the recording process, compress the first frames of image data, without previously performing gamma correction processing thereon, to generate the compressed image data stored on the second storage medium. ### Claim 20 (depends on 1) The imaging apparatus of claim 1, wherein the at least one first storage medium is further encoded with executable instructions that, when executed by the at least one processor, cause the at least one processor to control the circuitry of the apparatus so as to cause the circuitry to: in the recording process, compress the first frames of image data, without previously performing YC conversion processing thereon, to generate the compressed image data stored on the second storage medium. ### Claim 21 (depends on 1) The imaging apparatus of claim 1, wherein the at least one first storage medium is further encoded with executable instructions that, when executed by the at least one processor, cause the at least one processor to control the circuitry of the apparatus so as to cause the circuitry to: in the recording process, compress the first frames of image data by encoding each frame according to a predetermined format for each frame to generate the compressed image data stored on the second storage medium. ### Claim 22 (depends on 21) The imaging apparatus of claim 21, wherein the predetermined format for each frame is JPEG (Joint Photographic Experts Group) format. ### Claim 23 (depends on 1) The imaging apparatus of claim 1, wherein the at least one first storage medium is further encoded with executable instructions that, when executed by the at least one processor, cause the at least one processor to control the circuitry of the apparatus such that, in the recording process, the image data that is stored on the second storage medium is RAW image data. ### Claim 24 (depends on 1) The imaging apparatus of claim 1, wherein the color filter array comprises an RGB color filter array. ### Claim 25 (depends on 1) The imaging apparatus of claim 1, wherein the at least one first storage medium is further encoded with executable instructions that, when executed by the at least one processor, cause the at least one processor to control the circuitry of the apparatus so as to cause the circuitry to: in the recording process, generate the first signal by performing white balance processing on the second frames of image data at the second frame rate. ### Claim 26 (depends on 1) The imaging apparatus of claim 1, wherein the at least one first storage medium is further encoded with executable instructions that, when executed by the at least one processor, cause the at least one processor to control the circuitry of the apparatus so as to cause the circuitry to: in the recording process, generate the first signal by performing gamma correction processing on the second frames of image data at the second frame rate. ### Claim 27 (depends on 1) The imaging apparatus of claim 1, wherein the at least one first storage medium is further encoded with executable instructions that, when executed by the at least one processor, cause the at least one processor to control the circuitry of the apparatus so as to cause the circuitry to: in the recording process, generate the first signal by performing YC conversion processing on the second frames of image data at the second frame rate. ### Claim 28 (depends on 1) The imaging apparatus of claim 1, wherein the at least one processor comprises a microcomputer. ### Claim 29 An imaging apparatus, comprising: an image sensor that includes a color filter array and is configured to output at a first frame rate a first number of first frames of image data that have not yet been subjected to white balance processing, gamma correction processing, and YC conversion processing; at least one processor; and at least one first storage medium encoded with executable instructions that, when executed by the at least one processor, cause the at least one processor to control circuitry of the apparatus so as to cause the circuitry to: in a recording process, store the first frames of image data, without performing white balance processing, gamma correction processing, and YC conversion processing thereon, on a second storage medium while the second storage medium is attached to the imaging apparatus, the second storage medium being configured to be detachable from the imaging apparatus as to allow the image data to be processed by a device separate from the imaging apparatus when the second storage medium is detached from the imaging apparatus; and generate a first signal representing a first white-balanced, gamma-corrected, and YC converted image sequence that is to be displayed on a display as a camera-through image by processing the first frames of image data to generate a second number of second frames of image data that is lower than the first number of first frames of image data, and by performing at least one of white balance processing, gamma correction processing, or YC conversion processing on the second frames of image data at a second frame rate which is lower than the first frame rate. ### Claim 30 An method for operating an imaging apparatus, comprising: receiving, from an image sensor that includes a color filter array, at a first frame rate a first number of first frames of image data that have not yet been subjected to white balance processing, gamma correction processing, and YC conversion processing; in a recording process, compressing the first frames of image data, without previously performing white balance processing, gamma correction processing, and YC conversion processing thereon, to generate compressed image data; storing the compressed image data, without performing white balance processing, gamma correction processing, and YC conversion processing thereon, on a storage medium while the storage medium is attached to the imaging apparatus, the storage medium being configured to be detachable from the imaging apparatus as to allow the image data to be processed by a device separate from the imaging apparatus when the storage medium is detached from the imaging apparatus; and generating a first signal representing a first white-balanced, gamma-corrected, and YC converted image sequence that is to be displayed on a display as a camera-through image by processing the first frames of image data to generate a second number of second frames of image data that is lower than the first number of first frames of image data, and by performing at least one of white balance processing, gamma correction processing, or YC conversion processing on the second frames of image data at a second frame rate which is lower than the first frame rate.