# US 2016/0259099 A1 · Sensing images and light sources

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0259099 A1
- Document ID: US20160259099A1
- Status: Published application
- Kind: A1
- Application number: 14/638,302
- Filed: 2015-03-04
- Published: 2016-09-08
- Earliest claimed: 2015-03-04
- Assignees: Microsoft Technology Licensing, LLC
- Applicants: Microsoft Technology Licensing, LLC
- Inventors: Chengwu CUI
- Web page: https://patentgrep.com/patent/US20160259099A1

## CPC

- G01J 3/36: Investigating two or more bands of a spectrum by separate detectors
- G01J 3/505: {measuring the colour produced by lighting fixtures other than screens, monitors, displays or CRTs}
- G02B 5/204: {in which spectral selection is performed by means of a conductive grid or array, e.g. frequency selective surfaces (for use with wavelengths longer than the infrared light H01Q15/0006)}
- G06F 18/22: Matching criteria, e.g. proximity measures
- H04N 23/88: for colour balance, e.g. white-balance circuits or colour temperature control
- H04N 25/134: {based on three different wavelength filter elements}
- H04N 25/60: Noise processing, e.g. detecting, correcting, reducing or removing noise
- H04N 25/703: SSIS architectures incorporating pixels for producing signals other than image signals
- G01J 2001/4247: {for testing lamps or other light sources}
- G01J: MEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY (light sources F21, H01J, H01K, H05B; investigating properties of materials by optical means G01N)
- G02B: OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

The description relates to capturing or sensing color images of scenes and information about the type of light (e.g., light source) that illuminated the scene. One example can include an image sensor manifest as an array of pixels. This example can also include a lens configured to focus an image from a scene on the array of pixels. The array of pixels can entail first pixels that are configured to sense the image and second pixels that are configured to capture information about lighting of the scene.

## Claims

### Claim 1

A device, comprising:
an image sensor comprising an array of pixels;
a filter array positioned over the array of pixels; and,
the filter array comprising a first type of filter positioned over a majority of the pixels and a second type of filter positioned over a minority of the pixels, the first type of filter is configured to provide light filtering so that specific colors of human perceptible light from a visible spectrum are transmitted to underlying pixels and captured as color information of a sensed color image of a scene, and wherein the second type of filter comprises spectral band filters that transmit other portions of the visible spectrum to different underlying pixels that do not contribute color information to the sensed color image but capture other visible spectrum information.

### Claim 2 (depends on 1)

The device of claim 1, wherein the spectral band filters are selected to capture bandwidths that are indicative of specific types of light sources.

### Claim 3 (depends on 1)

The device of claim 1, wherein the colors of the first type of filter are associated with transmission wavelengths that overlap transmission wavelengths of the second type of filter.

### Claim 4 (depends on 1)

The device of claim 1, wherein the colors of the first type of filter are associated with transmission wavelengths that do not overlap transmission wavelengths of the second type of filter.

### Claim 5 (depends on 1)

The device of claim 1, further comprising a light source identification component configured to receive information from the image sensor, and wherein the light source identification component is further configured to distinguish other visible spectrum information from the color information of the received information.

### Claim 6 (depends on 5)

The device of claim 5, wherein the light source identification component is further configured to evaluate a profile of the other visible spectrum information to identify a light source of the scene.

### Claim 7

A device implemented process, comprising:
receiving information from pixels of an image sensor in response to the image sensor sensing a color image of a scene, the sensed color image of the scene based at least upon a set of colors and wherein a majority of the pixels are configured to detect one color of the set of colors;
identifying other pixels that are not dedicated to detecting one of the set of colors;
organizing the other pixels into groups based at least upon wavelength ranges that individual of the other pixels are configured to sense;
utilizing the information from the groups to identify a type of light source illuminating the scene;
processing the sensed color image based upon the type of light source illuminating the scene to produce an enhanced image; and,
causing the enhanced image to be displayed for a user.

### Claim 8 (depends on 7)

The device implemented process of claim 7, wherein the receiving information comprises receiving signals output from the pixels.

### Claim 9 (depends on 7)

The device implemented process of claim 7, wherein the identifying comprises identifying the other pixels from information from the image sensor about relative locations and filter type information of individual pixels on the image sensor.

### Claim 10 (depends on 7)

The device implemented process of claim 7, wherein the organizing further comprises, within an individual group, identifying pixels that are outliers, discarding the outliers, and evaluating remaining pixels of the group.

### Claim 11 (depends on 10)

The device implemented process of claim 10, wherein the evaluating comprises evaluating relative intensity of the information from the individual remaining pixels at individual wavelengths.

### Claim 12 (depends on 7)

The device implemented process of claim 7, wherein the utilizing comprises utilizing the information collectively to generate a spectral profile of visible wavelengths illuminating the scene.

### Claim 13 (depends on 7)

The device implemented process of claim 7, wherein the utilizing comprises utilizing the information to detect peaks representing spectral banding.

### Claim 14 (depends on 13)

The device implemented process of claim 13, further comprising utilizing a presence of the peaks at specific wavelengths to identity the light source.

### Claim 15 (depends on 7)

The device implemented process of claim 7, wherein the utilizing further comprises:
identifying gray regions of the sensed color image;
identifying whether any of the other pixels are in the gray regions;
determining a ratio of a signal profile of an individual other pixel compared to signal profiles of individual pixels in an individual gray region;
comparing the ratio to known ratios produced by known light source types to identify the type of light source.

### Claim 16 (depends on 15)

The device implemented process of claim 15, wherein the comparing comprises accessing the known ratios in a data table and determining a similarity between the ratio and individual known ratios.

### Claim 17 (depends on 16)

The device implemented process of claim 16, wherein the comparing identifies a match when the similarity exceeds a predefined threshold.

### Claim 18

A device, comprising:
an image sensor comprising an array of pixels;
a lens configured to focus an image from a scene on the array of pixels; and,
the array of pixels comprising first pixels that are configured to capture the image and second pixels that are configured to capture information about visible light of the scene.

### Claim 19 (depends on 18)

The device of claim 18, wherein the image sensor comprises a charge-coupled device (CCD) or a complementary metal-oxide semiconductor sensor (CMOS).

### Claim 20 (depends on 18)

The device of claim 18, manifest as a smart phone, a tablet, a wearable smart device, a single-lens reflex camera, a viewfinder camera, or a consumer device.
