# US 2022/0091265 A1 · Mobile robot generating resized region of interest in image frame and using dual-bandpass filter

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2022/0091265 A1
- Document ID: US20220091265A1
- Status: Published application
- Kind: A1
- Application number: 17/533,585
- Filed: 2021-11-23
- Published: 2022-03-24
- Earliest claimed: 2017-06-02
- Assignees: Pixart Imaging Inc.
- Applicants: PixArt Imaging Inc.
- Inventors: MIAN-JHONG CHIU; GUO-ZHEN WANG
- Web page: https://patentgrep.com/patent/US20220091265A1

## CPC

- A47L 11/4011: {Regulation of the cleaning machine by electric means; Control systems and remote control systems therefor}
- G01S 17/66: Tracking systems using electromagnetic waves other than radio waves
- G01B 11/25: by projecting a pattern, e.g. {one or more lines,} moiré fringes on the object (G01B11/255 takes precedence {; image analysis for depth or shape recovery G06T7/50})
- G01B 11/2522: {the position of the object changing and being recorded}
- G01B 21/047: {Accessories, e.g. for positioning, for tool-setting, for measuring probes}
- G01S 7/4808: {Evaluating distance, position or velocity data}
- G01S 7/4815: {using multiple transmitters}
- G01S 7/4816: {of receivers alone}
- G01S 17/46: Indirect determination of position data
- G01S 17/50: Systems of measurement based on relative movement of target
- G01S 17/931: of land vehicles
- G05D 1/0238: {using obstacle or wall sensors (G05D1/0246 and G05D1/0289 take precedence; lidar systems designed for anti-collision purposes G01S17/93)}
- G05D 1/0248: {in combination with a laser (lasers per se H01S)}
- G05D 1/0274: {using mapping information stored in a memory device (navigation using map-matching G01C21/30)}
- G06T 7/521: from laser ranging, e.g. using interferometry; from the projection of structured light
- G06V 10/14: Optical characteristics of the device performing the acquisition or on the illumination arrangements
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- H04N 23/20: for generating image signals from infrared radiation only
- H04N 23/51: Housings
- H04N 23/56: provided with illuminating means
- A47L 2201/04: Automatic control of the travelling movement; Automatic obstacle detection
- G01S 17/08: for measuring distance only (indirect measurement G01S17/46; active triangulation systems G01S17/48)
- G01S 17/48: Active triangulation systems, i.e. using the transmission and reflection of electromagnetic waves other than radio waves
- G06T 2207/10004: Still image; Photographic image
- G06T 2207/10152: Varying illumination
- H04N 23/57: Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
- A47L: DOMESTIC WASHING OR CLEANING (brushes A46B; cleaning quantities of bottles or of other hollow articles of one kind B08B9/00; laundry D06F); SUCTION CLEANERS IN GENERAL (cleaning in general B08)
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01B: MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

There is provided a mobile robot that performs the obstacle avoidance, positioning and object recognition according to image frames captured by the same optical sensor. The mobile robot includes an optical sensor, a light emitting diode, a laser diode and a processor. The processor identifies an obstacle and a distance thereof according to image frames captured by the optical sensor when the laser diode is emitting light. The processor further performs the positioning and object recognition according to image frames captured by the optical sensor when the light emitting diode is emitting light.

## Claims

### Claim 1

A mobile robot, comprising:
an optical sensor, configured to capture an image frame; and
a processor, electrically coupled to the optical sensor, and configured to
determine a region of interest (ROI) in the image frame,
extend a size of the ROI from an edge of the ROI to an integer times of a predetermined size to obtain an extended ROI, and
resize the extended ROI to the predetermined size.

### Claim 2 (depends on 1)

The mobile robot as claimed in claim 1, further comprising:
a first light source, configured to project a transverse light section toward a moving direction at a first time interval;
a second light source, configured to project a longitudinal light section toward the moving direction at a second time interval; and
a third light source, configured to illuminate a front area of the moving direction at a third time interval, wherein
the optical sensor is configured to respectively capture a first image frame, a second image frame and the image frame within the first time interval, the second time interval and the third time interval, and
the processor is configured to determine the ROI in the image frame according to at least one of the first image frame and the second image frame.

### Claim 3 (depends on 2)

The mobile robot as claimed in claim 2, wherein the optical sensor comprises a pixel array comprising a plurality of first pixels and a plurality of second pixels, the plurality of first pixels is configured to receive incident light via an IR light filter, and the plurality of second pixels is configured to receive incident light without via any light filter, wherein
the first image frame and the second image frame are formed by pixel data generated by the plurality of first pixels; and
the third image frame is generated by pixel data generated by both the plurality of first pixels and the plurality of second pixels.

### Claim 4 (depends on 3)

The mobile robot as claimed in claim 3, wherein the plurality of first pixels and the plurality of second pixels are arranged in a chessboard pattern.

### Claim 5 (depends on 1)

The mobile robot as claimed in claim 1, wherein the processor is configured to incorporate at least one of pixel rows and pixel columns adjacent to the ROI in the image frame with the ROI to obtain the extended ROI.

### Claim 6 (depends on 5)

The mobile robot as claimed in claim 5, wherein upon one side of the ROI being at an edge of the image frame, the processor is configured to incorporate the pixel rows or the pixel columns only adjacent to a side of the ROI opposite to the one side with the ROI to obtain the extended ROI.

### Claim 7 (depends on 5)

The mobile robot as claimed in claim 5, wherein the processor is configured to
incorporate a same number of pixel rows adjacent to two opposite first sides of the ROI with the ROI to obtain the extended ROI, and
incorporate a same number of pixel columns adjacent to two opposite second sides of the ROI with the ROI to obtain the extended ROI.

### Claim 8 (depends on 1)

The mobile robot as claimed in claim 1, wherein
the predetermined size is N×M,
the integer times is (p×N)×(q×M), wherein p is identical to or different from q, and
the processor is configured to sample one pixel every p pixels in an N-size direction, and sample one pixel every q pixels in an M-size direction in resizing the extended ROI.

### Claim 9 (depends on 8)

The mobile robot as claimed in claim 8, wherein the processor is configured to sample the one pixel from a first pixel of the ROI.

### Claim 10 (depends on 8)

The mobile robot as claimed in claim 8, wherein the processor is configured to sample the one pixel from a first pixel of the extended ROI.

### Claim 11

A mobile robot, comprising:
an optical sensor, configured to capture an image frame; and
a processor, electrically coupled to the optical sensor, and configured to
determine a region of interest (ROI) in the image frame, and
extend a size of the ROI from an edge of the ROI to an integer times of a predetermined size to obtain an extended ROI.

### Claim 12 (depends on 11)

The mobile robot as claimed in claim 11, further comprising:
a first light source, configured to project a transverse light section toward a moving direction at a first time interval;
a second light source, configured to project a longitudinal light section toward the moving direction at a second time interval; and
a third light source, configured to illuminate a front area of the moving direction at a third time interval, wherein
the optical sensor is configured to respectively capture a first image frame, a second image frame and the image frame within the first time interval, the second time interval and the third time interval, and
the processor is configured to determine the ROI in the image frame according to at least one of the first image frame and the second image frame.

### Claim 13 (depends on 12)

The mobile robot as claimed in claim 12, wherein the optical sensor comprises a pixel array comprising a plurality of first pixels and a plurality of second pixels, the plurality of first pixels is configured to receive incident light via an IR light filter, and the plurality of second pixels is configured to receive incident light without via any light filter, wherein
the first image frame and the second image frame are formed by pixel data generated by the plurality of first pixels; and
the third image frame is generated by pixel data generated by both the plurality of first pixels and the plurality of second pixels.

### Claim 14 (depends on 13)

The mobile robot as claimed in claim 13, wherein the plurality of first pixels and the plurality of second pixels are arranged in a chessboard pattern.

### Claim 15 (depends on 11)

The mobile robot as claimed in claim 11, wherein the processor is configured to incorporate at least one of pixel rows and pixel columns adjacent to the ROI in the image frame with the ROI to obtain the extended ROI.

### Claim 16 (depends on 15)

The mobile robot as claimed in claim 15, wherein upon one side of the ROI being at an edge of the image frame, the processor is configured to incorporate the pixel rows or the pixel columns only adjacent to a side of the ROI opposite to the one side with the ROI to obtain the extended ROI.

### Claim 17 (depends on 15)

The mobile robot as claimed in claim 15, wherein the processor is configured to
incorporate a same number of pixel rows adjacent to two opposite first sides of the ROI with the ROI to obtain the extended ROI, and
incorporate a same number of pixel columns adjacent to two opposite second sides of the ROI with the ROI to obtain the extended ROI.

### Claim 18

A mobile robot, comprising:
an optical sensor, configured to capture an image frame; and
a processor, electrically coupled to the optical sensor, and configured to
determine a region of interest (ROI) in the image frame,
calculate a ratio of a size of the ROI with respect to a predetermined size which is smaller than the size of the ROI, and
sample the ROI to resize the ROI to the predetermined size,
wherein the predetermined size is N×M, a first ratio in an N-size direction is an integer p, a second ratio in an M-size direction is an integer q, and the processor is configured to sample one pixel every p pixels in the N-size direction, and sample one pixel every q pixels in the M-size direction.

### Claim 19 (depends on 19)

The mobile robot as claimed in claim 19, further comprising:
a first light source, configured to project a transverse light section toward a moving direction at a first time interval; and
a third light source, configured to illuminate a front area of the moving direction at a third time interval, wherein
the optical sensor is configured to respectively capture a first image frame and the image frame within the first time interval and the third time interval, and
the processor is configured to determine the ROI in the image frame according to the first image frame.

### Claim 20 (depends on 19)

The mobile robot as claimed in claim 19, further comprising:
a second light source, configured to project a longitudinal light section toward the moving direction at a second time interval; and
a third light source, configured to illuminate a front area of the moving direction at a third time interval, wherein
the optical sensor is configured to respectively capture a second image frame and the image frame within the second time interval and the third time interval, and
the processor is configured to determine the ROI in the image frame according to the second image frame.

### Claim 21

A mobile robot, comprising:
a linear light source, configured to project a linear light section toward a moving direction of the mobile robot;
an optical sensor, configured to capture a bright image frame upon the linear light source being turned on and a dark image frame upon the linear light source being turned off;
a dual-bandpass filter, arranged at a light incident path of the optical sensor; and
a processor, electronically coupled to the linear light source and the optical sensor, and configured to
calculate a differential image frame between the bright image frame and dark image frame,
perform range estimation using the differential image frame; and
perform visual simultaneous localization and mapping (VSLAM) or image recognition using the dark image frame or another dark image frame captured by the optical sensor.

### Claim 22 (depends on 21)

The mobile robot as claimed in claim 21, wherein the linear light source is configured to project a transverse light section or a longitudinal light section.

### Claim 23 (depends on 21)

The mobile robot as claimed in claim 21, wherein the dual-bandpass filter is an IR and visible light pass filter.

### Claim 24 (depends on 21)

The mobile robot as claimed in claim 21, wherein the dual-bandpass filter is arranged at a part of or all of the light incident path of the optical sensor.

### Claim 25 (depends on 21)

The mobile robot as claimed in claim 21, wherein in the image recognition, the processor is configured to recognize an AprilTag as a virtual wall.

### Claim 26 (depends on 25)

The mobile robot as claimed in claim 25, further comprising a memory configured to record a distance of a ground line corresponding to a transverse light section projected by the linear light source, wherein the processor is further configured to
recognize the AprilTag only closer than the distance of the ground line.

### Claim 27 (depends on 25)

The mobile robot as claimed in claim 25, wherein the processor is further configured to control the mobile robot to perform different operations corresponding to different AprilTags.
