# US 12,183,069 B2 · Imaging apparatus, imaging system, imaging method, and imaging program > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,183,069 B2 - Document ID: US12183069B2 - Status: Granted - Kind: B2 - Application number: 18/353,893 - Filed: 2023-07-18 - Published: 2024-01-25 - Granted: 2024-12-31 - Earliest claimed: 2018-08-31 - Assignees: Sony Group Corporation - Applicants: Sony Group Corporation - Inventors: Ryuta Satoh; Suguru Aoki; Keitaro Yamamoto; Ryoji Eki - Web page: https://patentgrep.com/patent/US12183069B2 ## CPC - G06V 20/10: Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) - G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting - G06F 18/217: {Validation; Performance evaluation; Active pattern learning techniques} - G06N 3/044: Recurrent networks, e.g. Hopfield networks - G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] - G06N 3/045: Combinations of networks - G06N 3/0464: Convolutional networks [CNN, ConvNet] - G06N 3/08: Learning methods - G06N 3/0895: Weakly supervised learning, e.g. semi-supervised or self-supervised learning - G06N 3/09: Supervised learning - G06V 10/10: Image acquisition (document image scanning and transmission H04N1/00; control of digital cameras H04N23/60) - G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI] - G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle - H04N 23/60: Control of cameras or camera modules - H04N 23/61: based on recognised objects - H04N 23/611: where the recognised objects include parts of the human body - H04N 23/617: Upgrading or updating of programs or applications for camera control - H04N 23/667: Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes - H04N 23/73: by influencing the exposure time - H04N 23/80: Camera processing pipelines; Components thereof - H04N 25/441: by reading contiguous pixels from selected rows or columns of the array, e.g. interlaced scanning - H04N 25/443: by reading pixels from selected two-dimensional [2D] regions of the array, e.g. for windowing or digital zooming - H04N 25/445: by skipping some contiguous pixels within the read portion of the array - H04N 25/50: Control of the SSIS exposure - H04N 25/58: involving two or more exposures - H04N 25/745: {Circuitry for generating timing or clock signals} - H04N 25/75: Circuitry for providing, modifying or processing image signals from the pixel array - H04N 25/78: Readout circuits for addressed sensors, e.g. output amplifiers or A/D converters - H04N 25/79: Arrangements of circuitry being divided between different or multiple substrates, chips or circuit boards, e.g. stacked image sensors - H04N 25/531: by controlling rolling shutters in CMOS SSIS - H04N 25/532: by controlling global shutters in CMOS SSIS - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract An imaging apparatus includes: an imaging unit that has a pixel region in which a plurality of pixels is arranged and that reads and outputs a pixel signal from the pixels included in the pixel region; a unit-of-readout controller that controls a unit of readout set as a part of the pixel region: a recognition unit that has learned training data for each of units of readout; a first output unit that outputs a recognition result of recognition performed by the recognition unit for each of the units of readout; a second output unit that outputs the pixel signal to a subsequent stage; and a trigger generator that generates a trigger signal for controlling a first timing at which the first output unit outputs the recognition result and a second timing at which the second output unit outputs the pixel signal. ## Claims ### Claim 1 An imaging apparatus comprising: an image sensor that has a pixel region in which a plurality of pixels is arranged and that reads and outputs a pixel signal from the pixels included in the pixel region; at least one circuit configured to control a unit of readout set as a part of the pixel region: output a first recognition result of recognition for each of the units of readout; output the pixel signal to a subsequent stage; output a second recognition result of recognition for the pixel signal output to the subsequent stage; and generate a trigger signal for controlling a first timing at which the first recognition result is output, a second timing at which the pixel signal is output, and a third timing at which the second recognition result is output, wherein the at least one circuit is configured to instruct the image sensor to perform readout of the pixel signal for recognition from the image sensor at a rate higher than a rate of performing readout of the pixel signal from the image sensor to the subsequent stage, in accordance with an output of the trigger signal. ### Claim 2 (depends on 1) The imaging apparatus according to claim 1, wherein the trigger signal achieves synchronization between the first timing and the second timing. ### Claim 3 (depends on 1) The imaging apparatus according to claim 1, wherein the at least one circuit is configured to output the trigger signal at a fixed period. ### Claim 4 (depends on 1) The imaging apparatus according to claim 1, wherein the at least one circuit is configured to generate the trigger signal in accordance with external control of the imaging apparatus. ### Claim 5 (depends on 1) The imaging apparatus according to claim 1, wherein the at least one circuit is configured to generate the trigger signal in accordance with a recognition confidence indicating a confidence level of the first recognition result. ### Claim 6 (depends on 1) The imaging apparatus according to claim 1, wherein the at least one circuit is configured to control an output of the first recognition result and an output of the pixel signal, correct a deviation between the first recognition result output and the pixel signal output. ### Claim 7 (depends on 6) The imaging apparatus according to claim 6, wherein the at least one circuit is configured to hold the first recognition result output, and output the first recognition result held in synchronization with the output of the pixel signal. ### Claim 8 (depends on 7) The imaging apparatus according to claim 7, wherein the at least one circuit is configured to hold the pixel signal output, estimate a spatial deviation amount between the first recognition result and the pixel signal held, correct at least one of the first recognition result or the pixel signal held based on the estimated deviation amount, and output the first recognition result held and the corrected pixel signal held in synchronization with each other. ### Claim 9 (depends on 1) The imaging apparatus according to claim 1, wherein the at least one circuit is further configured to execute an image processing for the pixel signal output at the second timing, and generate a trigger signal for controlling a fourth timing at which image processing is executed. ### Claim 10 An imaging method executed by a processor, comprising: controlling a unit of readout set as a part of a pixel region in which a plurality of pixels is arranged, included in an image sensor: outputting a first recognition result of recognition for each of the units of readout; outputting a pixel signal read out from the pixel included in the pixel region to a subsequent stage; outputting a second recognition result of recognition for the pixel signal output to the subsequent stage; generating a trigger signal for controlling a first timing at which the first recognition result is output, a second timing at which the pixel signal is output, and a third timing at which the second recognition result is output; and instructing the image sensor to perform readout of the pixel signal for recognition from the image sensor at a rate higher than a rate of performing readout of the pixel signal from the image sensor to the subsequent stage, in accordance with an output of the trigger signal. ### Claim 11 (depends on 10) The imaging method according to claim 10, further comprising: image processing for the pixel signal output at the second timing, and generating a trigger signal for controlling a fourth timing at which image processing is executed. ### Claim 12 (depends on 10) The imaging method according to claim 10, wherein the processor is caused to execute: controlling an output of the first recognition result and an output of the pixel signal; and correcting a deviation between the first recognition result output and the pixel signal output. ### Claim 13 A non-transitory computer readable storage device having an imaging program that when executed by a processor causes the processor to execute: controlling a unit of readout set as a part of a pixel region in which a plurality of pixels is arranged, included in an image sensor: outputting a first recognition result of recognition for each of the units of readout; outputting a pixel signal read out from the pixel included in the pixel region to a subsequent stage; outputting a second recognition result of recognition for the pixel signal output to the subsequent stage; generating a trigger signal for controlling a first timing at which the first recognition result is output, a second timing at which the pixel signal is output, and a third timing at which the second recognition result is output; and instructing the image sensor to perform readout of the pixel signal for recognition from the image sensor at a rate higher than a rate of performing readout of the pixel signal from the image sensor to the subsequent stage, in accordance with an output of the trigger signal. ### Claim 14 (depends on 13) The non-transitory computer readable storage device according to claim 13, wherein the processor is caused to execute: image processing for the pixel signal output at the second timing, and generating a trigger signal for controlling a fourth timing at which image processing is executed. ### Claim 15 (depends on 13) The non-transitory computer readable storage device according to claim 13, wherein the processor is caused to execute: controlling an output of the first recognition result and an output of the pixel signal, correcting a deviation between the first recognition result output and the pixel signal output.