# US 2019/0320127 A1 · Configurable image processing system and methods for operating a configurable image processing system for multiple applications

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0320127 A1
- Document ID: US20190320127A1
- Status: Published application
- Kind: A1
- Application number: 15/952,719
- Filed: 2018-04-13
- Published: 2019-10-17
- Earliest claimed: 2018-04-13
- Assignees: Cornell University
- Applicants: Cornell University
- Inventors: Mark Buckler; Adrian Sampson; Suren Jayasuriya
- Web page: https://patentgrep.com/patent/US20190320127A1

## CPC

- H04N 25/42: by switching between different modes of operation using different resolutions or aspect ratios, e.g. switching between interlaced and non-interlaced mode
- H04N 25/00: Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N 25/445: by skipping some contiguous pixels within the read portion of the array
- H04N 25/76: Addressed sensors, e.g. MOS or CMOS sensors
- H04N 25/78: Readout circuits for addressed sensors, e.g. output amplifiers or A/D converters
- H04N 25/709: Circuitry for control of the power supply
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A configurable image processing system can process image data for multiple applications by including an image sensor capable of operating in a machine vision mode and a photography mode in response to an operating system command. When operating in machine vision mode, the image sensor may send image data to first processor for machine vision processing. When operating in photography mode, the image sensor may send image data to an image coprocessor for photography processing.

## Claims

### Claim 1

A configurable image processing system comprising:
at least one image sensor configured to operate in a machine vision mode and a photography mode in response to an operating system command, wherein the at least one image sensor is configured to send image data to a first processor while operating in machine vision mode, and further configured to send image data to an image coprocessor when operating in photography mode.

### Claim 2 (depends on 1)

The system of claim 1, wherein operating in photography mode includes operating in a higher resolution mode and operating in machine vision mode includes operating in a lower resolution mode.

### Claim 3 (depends on 1)

The system of claim 1, wherein operating in machine vision mode includes digitizing input image data by performing nonlinear quantization and operating in photography mode includes digitizing input image data by performing linear quantization.

### Claim 4 (depends on 3)

The system of claim 3, wherein performing nonlinear quantization includes performing logarithmic quantization.

### Claim 5 (depends on 1)

The system of claim 1, wherein operating in machine vision mode includes performing approximate demosaicing.

### Claim 6 (depends on 5)

The system of claim 5, wherein performing approximate demosaicing includes subsampling the input image data.

### Claim 7 (depends on 1)

The system of claim 1, wherein the image sensor further comprises a plurality of subpixel columns, each subpixel column in the plurality of subpixel columns comprising a plurality of subpixels, at least one analog-to-digital converter, at least one amplifier, and at least one switch.

### Claim 8 (depends on 7)

The system of claim 7, wherein the at least one analog-to-digital converter is a successive approximation analog-to-digital converter.

### Claim 9 (depends on 7)

The system of claim 7, wherein the at least one analog-to-digital converter is configured to perform linear quantization when the image sensor is operating in photography mode, and perform logarithmic quantization when the image sensor is operating in machine vision mode.

### Claim 10 (depends on 1)

The system of claim 1, wherein the image coprocessor is configured to perform demosaicing, denoising, color mapping, tone mapping, and compression, or a combination thereof.

### Claim 11 (depends on 1)

The system of claim 1, wherein the first processor comprises an artificial intelligence processor.

### Claim 12 (depends on 1)

The system of claim 1, wherein the first processor comprises a central processing unit.

### Claim 13

A method of operating a configurable image processing system for multiple applications comprising:
receiving, by an image sensor, instructions to operate in either a photography mode or a machine vision mode;
in response to receiving instructions to operate in a photography mode, capturing, by the image sensor, image data and sending the image data to an image coprocessor for image processing; and
in response to receiving instructions to operate in a machine vision mode, capturing, by the image sensor, image data and sending the image data to a first processor for image processing.

### Claim 14 (depends on 13)

The method of claim 13, wherein in response to receiving instructions to operate in a photography mode, the image sensor capture image data at a higher resolution, and wherein in response to receiving instructions to operate in a machine vision mode, the image sensor captures image data at a lower resolution.

### Claim 15 (depends on 13)

The method of claim 13, wherein in response to receiving instructions to operate in a photography mode, the image sensor digitizes the image data using linear quantization, and in response to receiving instructions to operate in a machine vision mode, the image sensor digitizes the image data using logarithmic quantization.

### Claim 16 (depends on 13)

The method of claim 13, wherein in response to receiving instructions to operate in a machine vision mode, the image sensor executes approximate demosaicing on the image data.

### Claim 17 (depends on 16)

The method of claim 16, wherein executing approximate demosaicing includes subsampling the image data.

### Claim 18

A method for operating a configurable image processing system for multiple applications comprising:
determining, by an operating system executing on processor first processor, whether an image capture application executing on the processor comprises a photography application or a machine vision application;
in response to determining that the image capture application comprises a photography application, sending, by the operating system, a photography mode control signal to an image sensor controller, and receiving by the operating system image data from an image coprocessor after the image coprocessor has processed image data captured by the image sensor in response to receiving the photography mode control signal;
in response to determining that the image capture application comprises a machine vision application:
sending, by the operating system, a machine vision mode control signal to the image sensor controller; and
receiving by the operating system image data captured by the image sensor in response to the machine vision control mode control signal without such image data being processed by the image coprocessor.

### Claim 19 (depends on 18)

The method of claim 18, wherein in response to determining that the image capture application comprises a machine vision application, the operating system sends a power-down control signal to the image coprocessor.

### Claim 20 (depends on 18)

The method of claim 18, wherein in response to determining that the image capture application comprises a machine vision application, the operating system sends the received image data to an artificial intelligence coprocessor.

### Claim 21 (depends on 18)

The method of claim 18, wherein in response to determining that the image capture application comprises a machine vision application, the operating system passes the received image data to the machine vision application.

### Claim 22 (depends on 18)

The method of claim 18, wherein the image data received by the operating system, in response to determining the application comprises a machine vision application, comprises lower resolution image data than the image data received by the operating system in response to determining the application comprises a photograph application.

### Claim 23 (depends on 18)

The method of claim 18, wherein the image data received by the operating system, in response to determining the application comprises a machine vision application, comprises non-linearly quantized image data, and the image data received by the operating system in response to determining the application comprises a photography application comprises linearly quantized image data.

### Claim 24

A computer-readable medium comprising instructions for an image capture application, which when executed by a processor comprising an operating system, causes the operating system to:
determine whether the image capture application executing on the processor comprises a photography application or a machine vision application;
in response to determining that the image capture application comprises a photography application, send a photography mode control signal to an image sensor controller, and receive image data from an image coprocessor after the image coprocessor has processed image data captured by the image sensor in response to receiving the photography mode control signal;
in response to determining that the image capture application comprises a machine vision application:
send a machine vision mode control signal to the image sensor controller; and
receive image data captured by the image sensor in response to the machine vision control mode control signal without such image data being processed by the image coprocessor.

### Claim 25 (depends on 24)

The computer readable medium of claim 24, wherein in response to determining that the image capture application comprises a machine vision application, the operating system sends a power-down control signal to the image coprocessor.

### Claim 26 (depends on 24)

The computer readable medium of claim 24, wherein in response to determining that the image capture application comprises a machine vision application, the operating system sends the received image data to an artificial intelligence coprocessor.

### Claim 27 (depends on 24)

The computer readable medium of claim 24, wherein in response to determining that the image capture application comprises a machine vision application, the operating system passes the received image data to the machine vision application.

### Claim 28 (depends on 24)

The computer readable medium of claim 24, wherein the image data received by the operating system, in response to determining the application comprises a machine vision application, comprises lower resolution image data than the image data received by the operating system in response to determining the application comprises a photography application.

### Claim 29 (depends on 24)

The computer readable medium of claim 24, wherein the image data received by the operating system, in response to determining the application comprises a machine vision application, comprises non-linearly quantized image data, and the image data received by the operating system in response to determining the application comprises a photograph application comprises linearly quantized image data.
