# US 12,597,231 B2 · Information processing device and information processing method

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,597,231 B2
- Document ID: US12597231B2
- Status: Granted
- Kind: B2
- Application number: 18/006,925
- Filed: 2021-07-29
- Published: 2023-09-14
- Granted: 2026-04-07
- Earliest claimed: 2020-08-17
- Assignees: Sony Group Corporation
- Applicants: Sony Group Corporation
- Inventors: Takeshi Ohashi
- Web page: https://patentgrep.com/patent/US12597231B2

## CPC

- G01S 17/894: Three-dimensional [3D] imaging with simultaneous measurement of time-of-flight at a two-dimensional [2D] array of receiver pixels, e.g. time-of-flight cameras or flash lidar
- G06V 10/761: {Proximity, similarity or dissimilarity measures}
- G01S 7/4802: {using analysis of echo signal for target characterisation; Target signature; Target cross-section}
- G01S 7/497: Means for monitoring or calibrating
- G06T 7/73: using feature-based methods
- G06T 17/00: Three-dimensional [3D] modelling for computer graphics
- G06V 10/143: Sensing or illuminating at different wavelengths
- G06V 10/147: Details of sensors, e.g. sensor lenses (fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19)
- G06V 10/25: Determination of region of interest [ROI] or a volume of interest [VOI]
- H04N 23/11: for generating image signals from visible and infrared light wavelengths
- G06T 2207/10048: Infrared image
- G06T 2207/20044: Skeletonization; Medial axis transform
- G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

Provided is an information processing device that includes an image acquisition unit, a measurement unit, and a recognition processing unit. The image acquisition unit acquires an image by receiving external light by a light receiving unit. The measurement unit measures a distance to a subject by using a first image based on the image acquired by the image acquisition unit in response to emission of infrared light from a light emitting unit. The recognition processing unit performs subject recognition processing by using a second image based on an image acquired by the image acquisition unit that has received external light without infrared light emitted from the light emitting unit.

## Claims

### Claim 1

An information processing device, comprising:
a camera that includes a light emitting diode (LED) and an image sensor, wherein
the image sensor is configured to receive external light,
the camera is configured to acquire a first image based on the received external light,
the LED is configured to emit infrared light, and
the camera is configured to acquire a second image based on the emitted infrared light; and
a central processing unit (CPU) configured to:
determine an illuminance of the external light is one of:
equal to or greater than a threshold, or
less than the threshold;
generate a third image from the second image;
perform a measurement process, based on the generated third image and the determination that the illuminance of the external light is less than the threshold, to measure a distance to a subject;
generate a fourth image from the first image; and
perform a subject recognition process based on the generated fourth image and the determination that the illuminance of the external light is equal to or greater than the threshold.

### Claim 2 (depends on 1)

The information processing device according to claim 1, wherein the CPU is further configured to perform the measurement process based on an analysis result of the third image.

### Claim 3 (depends on 2)

The information processing device according to claim 2, further comprising a battery configured to supply a remaining amount of power, wherein
the CPU is further configured to perform the measurement process based on the remaining amount of the power.

### Claim 4 (depends on 3)

The information processing device according to claim 3, wherein the CPU is further configured to:
transmit use purpose information indicating a use purpose of a subject recognition result of the subject recognition process; and
perform the measurement process based on the transmitted use purpose information.

### Claim 5 (depends on 1)

The information processing device according to claim 1, wherein the CPU is further configured to perform the subject recognition process based on the generated third image and the determination that the illuminance of the external light is less than the threshold.

### Claim 6 (depends on 1)

The information processing device according to claim 1, wherein the CPU is further configured to:
estimate a relative feature point position of the subject based on the generated fourth image;
generate a three-dimensional model of the subject based on a result of the measurement process; and
acquire an absolute feature point position of the subject based on the estimated relative feature point position of the subject and the generated three-dimensional model of the subject.

### Claim 7 (depends on 1)

The information processing device according to claim 1, wherein the CPU is further configured to:
acquire the generated fourth image at a first time;
estimate a feature point position of the subject based on the acquired fourth image;
acquire the third image at a second time closest to the first time at which the fourth image is acquired;
obtain distance information of a feature point of the subject from the acquired third image; and
acquire an absolute feature point position of the subject based on the estimated feature point position of the subject and the obtained distance information of the feature point of the subject.

### Claim 8

An information processing method, comprising:
in an information processing device that includes a camera and a central processing unit (CPU):
receiving, by an image sensor, external light, wherein the camera includes the image sensor;
acquiring, by the camera, a first image based on the received external light;
emitting, by a light emitting diode (LED), infrared light, wherein the camera includes the LED;
acquiring, by the camera, a second image based on the emitted infrared light;
determining, by the CPU, an illuminance of the external light is one of:
equal to or greater than a threshold, or
less than the threshold;
generating, by the CPU, a third image from the second image;
performing a measurement process, by the CPU, based on the generated third image and the determination that the illuminance of the external light is less than the threshold, to measure a distance to a subject;
generating, by the CPU, a fourth image from the first image; and
performing, by the CPU, a subject recognition process based on the generated fourth image and the determination that the illuminance of the external light is equal to or greater than the threshold.

### Claim 9

A non-transitory computer-readable medium having stored thereon, computer-executable instructions which, when executed by an information processing device, cause the information processing device to execute operations, the operations comprising:
receiving, by an image sensor, external light, wherein
the information processing device includes a camera and a central processing unit (CPU), and
the camera includes the image sensor and a light emitting diode (LED);
acquiring, by the camera, a first image based on the received external light;
emitting, by the LED, infrared light;
acquiring, by the camera, a second image based on the emitted infrared light;
determining, by the CPU, an illuminance of the external light is one of:
equal to or greater than a threshold, or
less than the threshold;
generating, by the CPU, a third image from the second image;
performing a measurement process, by the CPU, based on the generated third image and the determination that the illuminance of the external light is less than the threshold, to measure a distance to a subject;
generating, by the CPU, a fourth image from the first image; and
performing, by the CPU, a subject recognition process based on the generated fourth image and the determination that the illuminance of the external light is equal to or greater than the threshold.