# US 2021/0173400 A1 · Information processing device, information processing method, and computer readable medium

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2021/0173400 A1
- Document ID: US20210173400A1
- Status: Published application
- Kind: A1
- Application number: 16/475,576
- Filed: 2017-03-15
- Published: 2021-06-10
- Earliest claimed: 2017-03-15
- Assignees: Mitsubishi Electric Corporation
- Applicants: Mitsubishi Electric Corporation
- Inventors: Takefumi HASEGAWA; Masahiro ABUKAWA; Shu MURAYAMA; Yohei MIKI; Michinori YOSHIDA; Kentaro ISHIKAWA
- Web page: https://patentgrep.com/patent/US20210173400A1

## CPC

- G08G 1/005: including pedestrian guidance indicator
- B62D 15/0285: {Parking performed automatically}
- G05D 1/0214: {in accordance with safety or protection criteria, e.g. avoiding hazardous areas (monitoring the location of vehicles within a certain area, e.g. forbidden or allowed areas, in traffic control systems for road vehicles G08G1/13)}
- G05D 1/0246: {using a video camera in combination with image processing means}
- G05D 1/0297: {by controlling means in a control room}
- G08G 1/0116: {from roadside infrastructure, e.g. beacons}
- G08G 1/0145: {for active traffic flow control}
- G08G 1/04: using optical or ultrasonic detectors
- G08G 1/143: {inside the vehicles}
- G08G 1/146: {where the parking area is a limited parking space, e.g. parking garage, restricted space}
- H04N 7/18: Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})
- B62D: MOTOR VEHICLES; TRAILERS (steering, or guiding on a desired track, of agricultural machines or implements A01B69/00; wheels, castors, axles, increasing wheel adhesion B60B; vehicle tyres, tyre inflation or tyre changing B60C; connections between vehicles of a train or the like B60D; vehicles for use on rail and road, amphibious or convertible vehicles B60F; suspension arrangements B60G; heating, cooling, ventilating or other air treating devices B60H; windows, windscreens, non-fixed roofs, doors or similar devices, protective coverings for vehicles not in use B60J; propulsion plant arrangements, auxiliary drives, transmissions, controls, instrumentation or dashboards B60K; electric equipment or propulsion of electrically-propelled vehicles B60L; power supply for electrically-propelled vehicles B60M; passenger accommodation not otherwise provided for B60N; adaptations for load transportation or to carry special loads or objects B60P; arrangement of signalling or lighting devices, the mounting or supporting thereof or circuits therefor, for vehicles in general B60Q; vehicles, vehicle fittings or vehicle parts not otherwise provided for B60R; servicing, cleaning, repairing, supporting, lifting, or manoeuvring, not otherwise provided for, B60S; brake arrangements, brake control systems or parts thereof B60T; air-cushion vehicles B60V; motor-cycles, accessories therefor B62J, B62K; testing of vehicles G01M)
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A first image-data acquisition unit (101) acquires a plurality of pieces of image data in which different areas in a facility in which an autonomous driving vehicle conducts autonomous traveling are shown. A priority setting unit (102) analyzes a status shown in each of the plurality of pieces of image data and sets a priority to each image data. A first image-data display unit (103) decides a display mode at a time of displaying the plurality of pieces of image data according to the priority set by the priority setting unit (102).

## Claims

### Claim 1

An information processing device comprising:
processing circuitry
to acquire a plurality of pieces of image data in which different areas in a facility in which an autonomous driving vehicle conducts autonomous traveling are shown;
to analyze a status shown in each of the plurality of pieces of image data and to set a priority to each image data; and
to combine the plurality of pieces of image data to generate a bird's-eye view, and to highlight a portion of the bird's-eye view in which image data in which the priority set matches a predetermined condition is reflected.

### Claim 2 (depends on 1)

The information processing device according to claim 1,
wherein the processing circuitry analyzes a status shown in each image data, estimates a degree of danger in an area that is shown in each image data, and sets a priority to each image data based on an estimated degree of danger.

### Claim 3 (depends on 2)

The information processing device according to claim 2,
wherein the processing circuitry acquires a plurality of pieces of image data in which different areas in a facility in which one or more autonomous driving vehicles conduct autonomous driving and one or more pedestrians walk are shown, and
analyzes a status of an autonomous driving vehicle and a status of a pedestrian as a status shown in each image data, and estimates a degree of danger in an area that is shown in each image data.

### Claim 4

(canceled)

### Claim 5

(canceled)

### Claim 6 (depends on 1)

The information processing device according to claim 1,
wherein the processing circuitry acquires a plurality of pieces of image data generated by a plurality of sensor devices installed in a plurality of autonomous traveling vehicles that are present in the facility.

### Claim 7

An information processing device comprising:
processing circuitry
to acquire a plurality of pieces of facility-sensor image data being a plurality of pieces of image data in which different areas in a facility in which an autonomous driving vehicle conducts autonomous traveling are shown, the plurality of pieces of facility-sensor image data being generated by a plurality of sensor devices installed in the facility, and to acquire a plurality of pieces of vehicle-sensor image data being a plurality of pieces of image data generated by a plurality of sensor devices installed in a plurality of autonomous traveling vehicles that are present in the facility;
to analyze the plurality of pieces of facility-sensor image data and the plurality of pieces of vehicle-sensor image data, to estimate a degree of danger in an area that is shown in each facility-sensor image data, and to set a priority to each facility-sensor image data based on the estimated degree of danger; and
to decide a display mode at a time of displaying the plurality of pieces of facility-sensor image data according to the priority set.

### Claim 8

An information processing device comprising:
processing circuitry
to acquire a plurality of pieces of facility-sensor image data being a plurality of pieces of image data in which different areas in facility in which an autonomous driving vehicle conducts autonomous traveling are shown, the plurality of pieces of facility-sensor image data being generated by a plurality of sensor devices installed in the facility, and to acquire a plurality of pieces of vehicle-sensor image data being a plurality of pieces of image data generated by a plurality of sensor devices installed in a plurality of autonomous traveling vehicles that are present in the facility;
to analyze the plurality of pieces of facility-sensor image data and the plurality of pieces of vehicle-sensor image data, to estimate a degree of danger in an area that is shown in each facility-sensor image data, and to set a priority to each vehicle-sensor image data based on the estimated degree of danger; and
to decide a display mode at a time of displaying the plurality of pieces of vehicle-sensor image data according to the priority set.

### Claim 9 (depends on 1)

The information processing device according to claim 1,
wherein the processing circuitry instructs stopping of traveling of the autonomous driving vehicle.

### Claim 10 (depends on 1)

The information processing device according to claim 1,
wherein the processing circuitry designates a traveling route and instructs traveling of the autonomous driving vehicle along the traveling route.

### Claim 11 (depends on 1)

The information processing device according to claim 1,
wherein the processing circuitry instructs, when there are a plurality of autonomous driving vehicles in the facility, stopping of traveling of the plurality of autonomous driving vehicles.

### Claim 12

An information processing method comprising:
acquiring a plurality of pieces of image data in which different areas in a facility in which an autonomous driving vehicle conducts autonomous driving are shown;
analyzing a status shown in each of the plurality of pieces of image data and setting a priority to each image data; and
combining the plurality of pieces of image data to generate a bird's-eye view, and highlighting a portion of the bird's-eye view in which image data in which the priority set matches a predetermined condition is reflected.

### Claim 13

A non-transitory computer readable medium storing an information processing program that causes a computer to execute:
an image-data acquiring process of acquiring a plurality of pieces of image data in which different areas in a facility in which an autonomous driving vehicle conducts autonomous driving are shown;
a priority setting process of analyzing a status shown in each of the plurality of pieces of image data and setting a priority to each image data; and
a display control process of combining the plurality of pieces of image data to generate a bird's-eye view, and of highlighting a portion of the bird's-eye view in which image data in which the priority set by the priority setting process matches a predetermined condition is reflected.

### Claim 14

An information processing method comprising:
acquiring a plurality of pieces of facility-sensor image data being a plurality of pieces of image data in which different areas in a facility in which an autonomous driving vehicle conducts autonomous traveling are shown, the plurality of pieces of facility-sensor image data being generated by a plurality of sensor devices installed in the facility, and acquiring a plurality of pieces of vehicle-sensor image data being a plurality of pieces of image data generated by a plurality of sensor devices installed in a plurality of autonomous traveling vehicles that are present in the facility;
analyzing the plurality of pieces of facility-sensor image data and the plurality of pieces of vehicle-sensor image data, estimating a degree of danger in an area that is shown in each facility-sensor image data, and setting a priority to each facility-sensor image data based on the estimated degree of danger; and
deciding a display mode at a time of displaying the plurality of pieces of facility-sensor image data according to the priority set.

### Claim 15

An information processing method comprising:
acquiring a plurality of pieces of facility-sensor image data being a plurality of pieces of image data in which different areas in a facility in which an autonomous driving vehicle conducts autonomous traveling are shown, the plurality of pieces of facility-sensor image data being generated by a plurality of sensor devices installed in the facility, and acquiring a plurality of pieces of vehicle-sensor image data being a plurality of pieces of image data generated by a plurality of sensor devices installed in a plurality of autonomous traveling vehicles that are present in the facility;
analyzing the plurality of pieces of facility-sensor image data and the plurality of pieces of vehicle-sensor image data, estimating a degree of danger in an area that is shown in each facility-sensor image data, and setting a priority to each vehicle-sensor image data based on the estimated degree of danger; and
deciding a display mode at a time of displaying the plurality of pieces of vehicle-sensor image data according to the priority set.

### Claim 16

A non-transitory computer readable medium storing an information processing program that causes a computer to execute:
an image-data acquisition process of acquiring a plurality of pieces of facility-sensor image data being a plurality of pieces of image data in which different areas in a facility in which an autonomous driving vehicle conducts autonomous traveling are shown, the plurality of pieces of facility-sensor image data being generated by a plurality of sensor devices installed in the facility, and of acquiring a plurality of pieces of vehicle-sensor image data being a plurality of pieces of image data generated by a plurality of sensor devices installed in a plurality of autonomous traveling vehicles that are present in the facility;
a priority setting process of analyzing the plurality of pieces of facility-sensor image data and the plurality of pieces of vehicle-sensor image data, of estimating a degree of danger in an area that is shown in each facility-sensor image data, and of setting a priority to each facility-sensor image data based on the estimated degree of danger; and
a display control process of deciding a display mode at a time of displaying the plurality of pieces of facility-sensor image data according to the priority set by the priority setting process.

### Claim 17

A non-transitory computer readable medium storing an information processing program that causes a computer to execute:
an image-data acquisition process of acquiring a plurality of pieces of facility-sensor image data being a plurality of pieces of image data in which different areas in a facility in which an autonomous driving vehicle conducts autonomous traveling are shown, the plurality of pieces of facility-sensor image data being generated by a plurality of sensor devices installed in the facility, and of acquiring a plurality of pieces of vehicle-sensor image data being a plurality of pieces of image data generated by a plurality of sensor devices installed in a plurality of autonomous traveling vehicles that are present in the facility;
a priority setting process of analyzing the plurality of pieces of facility-sensor image data and the plurality of pieces of vehicle-sensor image data, of estimating a degree of danger in an area that is shown in each facility-sensor image data, and of setting a priority to each vehicle-sensor image data based on the estimated degree of danger; and
a display control process of deciding a display mode at a time of displaying the plurality of pieces of vehicle-sensor image data according to the priority set by the priority setting process.

### Claim 18 (depends on 7)

The information processing device according to claim 7,
wherein the processing circuitry instructs stopping of traveling of the autonomous driving vehicle.

### Claim 19 (depends on 7)

The information processing device according to claim 7,
wherein the processing circuitry designates a traveling route and instructs traveling of the autonomous driving vehicle along the traveling route.

### Claim 20 (depends on 7)

The information processing device according to claim 7,
wherein the processing circuitry instructs, when there are a plurality of autonomous driving vehicles in the facility, stopping of traveling of the plurality of autonomous driving vehicles.

### Claim 21 (depends on 8)

The information processing device according to claim 8,
wherein the processing circuitry instructs stopping of traveling of the autonomous driving vehicle.

### Claim 22 (depends on 8)

The information processing device according to claim 8,
wherein the processing circuitry designates a traveling route and instructs traveling of the autonomous driving vehicle along the traveling route.

### Claim 23 (depends on 8)

The information processing device according to claim 8,
wherein the processing circuitry instructs, when there are a plurality of autonomous driving vehicles in the facility, stopping of traveling of the plurality of autonomous driving vehicles.
