# US 2016/0239726 A1 · Vision sensor, system, and method for image processing, and vision based interactive device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2016/0239726 A1
- Document ID: US20160239726A1
- Status: Published application
- Kind: A1
- Application number: 15/017,285
- Filed: 2016-02-05
- Published: 2016-08-18
- Earliest claimed: 2015-02-12
- Assignees: Morpx Inc.
- Applicants: Morpx Inc.
- Inventors: Tianli YU
- Web page: https://patentgrep.com/patent/US20160239726A1

## CPC

- H04N 23/57: Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
- H04N 23/611: where the recognised objects include parts of the human body
- H04N 23/80: Camera processing pipelines; Components thereof
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A vision based sensor, image processing method and system, and an interactive device including the same are provided. The sensor includes an optical lens, an image acquisition chip that captures image resolution below a first threshold, and a micro-controller. The image acquisition chip acquires an object's image through the optical lens, and output an ultra-low resolution image with a resolution below the first threshold, and sends the ultra-low resolution image to the micro-controller. The micro-controller obtains object information by analyzing the ultra-low resolution image, and sends the object information to an external component, so that the external component performs a predefined act.

## Claims

### Claim 1

A vision sensor, comprising:
an optical lens, an image acquisition chip that captures image resolution below a first threshold, and a micro-controller;
wherein the image acquisition chip acquires an object's image through the optical lens, and output an ultra-low resolution image with a resolution below the first threshold, and sends the ultra-low resolution image to the micro-controller; and
wherein the micro-controller obtains object information by analyzing the ultra-low resolution image, and sends the object information to an external component, so that the external component performs a predefined act.

### Claim 2 (depends on 1)

The vision sensor of claim 1, wherein the first threshold corresponds to a resolution of 96×96, and the resolution of the image acquisition chip is above a second threshold; and wherein the second threshold is lower than the first threshold.

### Claim 3 (depends on 1)

The vision sensor in claim 1, wherein the micro-controller further comprises static random access memory (SRAM) and flash memory to store the ultra-low resolution image, wherein the SRAM has a storage size that is not greater than 32 KB and the flash memory has a storage size that is not greater than 256 KB.

### Claim 4 (depends on 1)

The vision sensor of claim 1, further comprising:
a power supply circuitry to power the image acquisition chip and micro-controller;
wherein the power supply circuitry comprises at least one of the following: a lithium battery, an alkaline battery, a button battery, and a solar cell.

### Claim 5

An image processing method, comprising:
acquiring an ultra-low resolution image with the resolution below a first threshold from an image acquisition chip;
obtaining object information by using a predefined algorithm to analyze the ultra-low resolution image in multiple scales, and
sending the object information to trigger a response based on the object information.

### Claim 6 (depends on 5)

The image processing method of claim 5, wherein obtaining object information by using a predefined algorithm to analyze the ultra-low resolution image in multiple scales comprises:
preprocessing the ultra-low resolution image from the image acquisition chip;
rescaling the preprocessed image into predefined scales;
using a predefined detection window to scan rescaled image at each predefined scale, and buffering pixel rows covered by the current position of the predefined detection window; and
obtaining object information by using a predefined algorithm to analyze the buffered pixel rows.

### Claim 7 (depends on 5)

The image processing method of claim 5, wherein the object information comprise at least one of:
human face detection information, human face recognition information, facial expression information, object tracking and recognition information, and environment recognition information.

### Claim 8 (depends on 7)

The image processing method of claim 7, wherein:
the human face detection information is configured to detect human face in a ultra-low resolution image;
the human face recognition information is configured to recognize a human face after a face is detected in a ultra-low resolution image;
the facial expression information is configured to recognize the facial expression after face detected in a ultra-low resolution image;
the object tracking and recognition information is configured to determine directional information of the recognized object in the ultra-low resolution image; and
the environment recognition information is used to recognize environment from the ultra-low resolution image.

### Claim 9

An apparatus for image processing, comprising:
a receiver circuitry configured to receive an ultra-low resolution image below a first threshold from an image acquisition chip;
an image processing circuitry configured to apply a predefined algorithm to apply multi-scale image processing to extract recognition information; and
a transmission circuitry configured to transmit the recognition information to an external component, so that the external component triggers a response.

### Claim 10 (depends on 9)

The apparatus of claim 9, wherein the image processing circuitry comprises:
a preprocessing unit, configured to preprocess the ultra-low resolution image acquired from the image acquisition chip;
an image scaling unit, configured to scale the ultra-low resolution image to obtain a scaled image at a predefined scale;
a scanning unit, configured to apply predefined detection window to scan the scaled image, and buffer pixel rows of the scaled image covered by the predefined detection window; and
a recognition unit, configured to apply the predefined algorithm to recognize object in the buffered pixel rows.

### Claim 11

An interactive device, comprising:
a storage device, a vision sensor, a controller component, and at least one of: a mechanical motion device, a display device, and a sound player device,
wherein the vision sensor comprises an optical lens, an image acquisition chip that captures image resolution below a first threshold, and a micro-controller;
wherein the image acquisition chip acquires an object's image through the optical lens, and output an ultra-low resolution image with a resolution below the first threshold, and sends the ultra-low resolution image to the micro-controller;
wherein the micro-controller obtains object information by analyzing the ultra-low resolution image, and sends the object information to the interactive device, so that the interactive device performs a predefined act; and
wherein the storage device stores the control parameter database including one or more control parameters correspond to various recognized object information.

### Claim 12 (depends on 11)

The interactive device of claim 11, wherein the controller device looks up the control parameter from the control parameter database that correspond to the recognized object information, and drive the controlled device to execute the control parameters.

### Claim 13 (depends on 11)

The interactive device of claim 11, wherein the first threshold corresponds to a resolution of 96×96, and the resolution of the image acquisition chip is above a second threshold; and wherein the second threshold is lower than the first threshold.

### Claim 14 (depends on 11)

The interactive device of claim 11, wherein the micro-controller further comprises static random access memory (SRAM) and flash memory to store the ultra-low resolution image, wherein the SRAM has a storage size that is not greater than 32 KB and the flash memory has a storage size that is not greater than 256 KB.

### Claim 15 (depends on 11)

The interactive device of claim 11, further comprising:
a power supply circuitry to power the image acquisition chip and micro-controller;
wherein the power supply circuitry comprises at least one of the following: a lithium battery, an alkaline battery, a button battery, and a solar cell.
