# US 12,214,492 B2 · Information processing device and storage medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,214,492 B2
- Document ID: US12214492B2
- Status: Granted
- Kind: B2
- Application number: 17/451,511
- Filed: 2021-10-20
- Published: 2022-02-03
- Granted: 2025-02-04
- Earliest claimed: 2016-06-14
- Assignees: Sony Group Corporation
- Applicants: Sony Group Corporation
- Inventors: Hiroshi Iwanami
- Web page: https://patentgrep.com/patent/US12214492B2

## CPC

- B25J 11/008: {Manipulators for service tasks}
- B25J 19/026: {Acoustical sensing devices}
- B25J 9/0003: {Home robots, i.e. small robots for domestic use}
- B25J 9/1666: {Avoiding collision or forbidden zones}
- B25J 9/1676: {Avoiding collision or forbidden zones}
- B25J 11/0005: {Manipulators having means for high-level communication with users, e.g. speech generator, face recognition means}
- G06N 99/00: Subject matter not provided for in other groups of this subclass
- G08B 25/04: using a single signalling line, e.g. in a closed loop
- B25J: MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES ({manipulators specially adapted for use in surgery A61B34/70; manipulators used in cleaning hollow articles B08B9/04}; manipulators associated with rolling mills B21B39/20; manipulators associated with forging machines B21J13/10; {manipulators associated with picking-up and placing mechanisms B23P19/007}; means for holding wheels or parts thereof B60B30/00; {vehicles with ground-engaging propulsion means, e.g. walking members B62D57/02, B62D57/032; devices for picking-up and depositing articles or materials between conveyors B65G47/90, B65G47/91; manipulators with gripping or holding means for transferring packages B65H67/065}; cranes B66C; {manipulators used in the protection or supervision of pipe-line installations F17D5/00; walking equipment adapted for nuclear steam-generators F22B37/006}; manipulators specially adapted for, or associated with, nuclear reactors G21C; {apparatus used for handling wafers during manufacture or treatment of semiconductor H10P72/50})
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS

## Abstract

There is provided an information processing device including a passive ultrasonic sensor, a notification section that notifies a user, and a control section that, when detecting a feature indicating approach of an object learned in advance from sensor data detected by the ultrasonic sensor, controls the notification section to notify the user.

## Claims

### Claim 1

An information processing apparatus, comprising:
circuitry configured to:
detect feature quantity data for each piece of sensor data detected in a chronological order by a passive ultrasonic sensor, wherein
the detected feature quantity data indicates an approach of an object, and
the approach of the object is learned in advance;
compare the detected feature quantity data for the each piece of the sensor data detected in the chronological order with teacher data stored in a storage section;
identify the object based on the comparison; and
control a notification to a user based on the identified object, wherein the notification indicates the approach of the object.

### Claim 2 (depends on 1)

The information processing apparatus according to claim 1, wherein
the circuitry is further configured to classify an event indicated by the detected feature quantity data for the each piece of the sensor data detected in the chronological order based on the comparison, and
each classified event is associated with the object.

### Claim 3 (depends on 2)

The information processing apparatus according to claim 2, wherein the circuitry is further configured to:
detect an order of transition of the classified event for the each piece of the sensor data detected in the chronological order; and
identify the object based on the detected order of transition.

### Claim 4 (depends on 1)

The information processing apparatus according to claim 1, further comprising the passive ultrasonic sensor configured to detect the sensor data.

### Claim 5 (depends on 1)

The information processing apparatus according to claim 1, wherein the circuitry is further configured to:
acquire a first command from the user;
recognize that the acquired first command is a specific command;
generate the teacher data based on the recognition that the acquired first command is the specific command, wherein
the teacher data is generated based on the sensor data detected in the chronological order since a first time point going back a specific time period, and
the teacher data is for recognition of the object approaching; and
store the generated teacher data in the storage section.

### Claim 6 (depends on 5)

The information processing apparatus according to claim 5, wherein
the teacher data indicates a feature of the sensor data detected in the chronological order since the first time point going back the specific time period.

### Claim 7 (depends on 5)

The information processing apparatus according to claim 5, wherein the circuitry is further configured to:
acquire a second command from the user based on the notification to the user;
recognize that the acquired second command indicates a request for an earlier notification; and
update the generated teacher data based on the recognition that the acquired second command indicates the request for the earlier notification, wherein
the teacher data is updated based on the feature quantity data of the sensor data detected in the chronological order since a second time point going back further than the specific time period, and
the second time point is before the first time point.

### Claim 8 (depends on 1)

The information processing apparatus according to claim 1, wherein the circuitry is further configured to:
detect that the feature quantity data is similar to the teacher data;
control the notification to the user based on the detection that the feature quantity data is similar to the teacher data, wherein the notification indicates the approach of the object;
acquire a second command from the user based on the notification to the user;
recognize that the acquired second command indicates a positive evaluation; and
update, based on the recognition that the second command indicates the positive evaluation, the teacher data generated based on the sensor data detected in the chronological order by the passive ultrasonic sensor.

### Claim 9 (depends on 1)

The information processing apparatus according to claim 1, wherein
the detected feature quantity data is a waveform.

### Claim 10 (depends on 1)

The information processing apparatus according to claim 1, further comprising a radio wave receiver configured to receive a radio wave signal.

### Claim 11 (depends on 10)

The information processing apparatus according to claim 10, wherein
the circuitry is further configured to:
detect the feature quantity data based on the sensor data detected by the passive ultrasonic sensor and the radio wave signal received by the radio wave receiver; and
control the notification to the user.

### Claim 12 (depends on 10)

The information processing apparatus according to claim 10, wherein the circuitry is further configured to:
execute a first recognition process to recognize the object based on an ID included in the received radio wave signal;
execute a second recognition process to recognize the object based on the detected feature quantity data, wherein the second recognition process is executed after the first recognition process; and
control the notification to the user based on a coincidence between a result of the first recognition process and a result of the second recognition process.

### Claim 13 (depends on 1)

The information processing apparatus according to claim 1, further comprising
a smell sensor configured to detect smell data, wherein
the circuitry is further configured to:
detect the feature quantity data based on the sensor data detected by the passive ultrasonic sensor and the smell data detected by the smell sensor; and
control the notification to the user.

### Claim 14 (depends on 13)

The information processing apparatus according to claim 13, wherein
the smell data and the object are associated in advance,
the sensor data and the object are associated in advance, and
the circuitry is further configured to:
execute a first recognition process to recognize the object based on the smell data;
execute a second recognition process to recognize the object based on the sensor data; and
control the notification to the user based on a coincidence between a result of the first recognition process and a result of the second recognition process.

### Claim 15 (depends on 1)

The information processing apparatus according to claim 1, further comprising
a camera configured to capture an image of a face of the user at a time when the user answers a visitor, wherein
the visitor serves as the object, and
the circuitry is further configured to:
estimate an emotion of the user from the captured image of the face of the user; and
update a content of the notification to the user based on a result of the estimation of the emotion of the user.

### Claim 16 (depends on 1)

The information processing apparatus according to claim 1, further comprising
a camera configured to capture an image of a face of the user at a first time point when the user answers a visitor, wherein
the visitor serves as the object, and
the circuitry is further configured to:
estimate an emotion of the user based on the captured image of the face of the user;
generate the teacher data based on a result of the estimation and the sensor data detected in the chronological order since a second time point going back a specific time period from the first time point, wherein the generated teacher data is for recognition of the visitor; and
store the generated teacher data in the storage section.

### Claim 17

An information processing method, comprising:
in an information processing apparatus comprising circuitry:
detecting feature quantity data for each piece of sensor data detected in a chronological order by a passive ultrasonic sensor, wherein
the detected feature quantity data indicates an approach of an object, and the approach of the object is learned in advance;
comparing the detected feature quantity data for the each piece of the sensor data detected in the chronological order with teacher data stored in a storage section;
identifying the object based on the comparison; and
controlling a notification to a user based on the identified object, wherein the notification indicates the approach of the object.

### Claim 18

A non-transitory computer-readable medium having stored thereon computer-executable instructions which, when executed by a processor of an information processing apparatus, cause the processor to execute operations, the operations comprising:
detecting feature quantity data for each piece of sensor data detected in a chronological order by a passive ultrasonic sensor, wherein
the detected feature quantity data indicates an approach of an object, and the approach of the object is learned in advance;
comparing the detected feature quantity data for the each piece of the sensor data detected in the chronological order with teacher data stored in a storage section;
identifying the object based on the comparison; and
controlling a notification to a user based on the identified object, wherein the notification indicates the approach of the object.