# US 2026/0219666 A1 · Information processing device, information processing method, and storage medium storing a program

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0219666 A1
- Document ID: US20260219666A1
- Status: Published application
- Kind: A1
- Application number: 19/641,527
- Filed: 2026-04-08
- Published: 2026-07-30
- Earliest claimed: 2023-10-17
- Assignees: SoftBank Group Corp.
- Applicants: SoftBank Group Corp.
- Inventors: Masayoshi SON
- Web page: https://patentgrep.com/patent/US20260219666A1

## CPC

- G05D 1/243: Means capturing signals occurring naturally from the environment, e.g. ambient optical, acoustic, gravitational or magnetic signals (using passive navigation aids external to the vehicle G05D1/244; using signals from positioning sensors located off-board the vehicle G05D1/249)
- G08G 1/166: {for active traffic, e.g. moving vehicles, pedestrians, bikes}
- B60W 10/04: including control of propulsion units
- B60W 10/22: including control of suspension systems
- B60W 10/30: including control of auxiliary equipment, e.g. air-conditioning compressors or oil pumps
- B60W 30/09: Taking automatic action to avoid collision, e.g. braking and steering
- B60W 30/0956: {the prediction being responsive to traffic or environmental parameters}
- G06T 7/593: from stereo images
- G06V 10/82: using neural networks
- G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
- G08G 1/04: using optical or ultrasonic detectors
- G08G 1/16: Anti-collision systems (road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08)
- G08G 1/163: {involving continuous checking}
- G08G 1/164: {Centralised systems, e.g. external to vehicles}
- G16Y 10/40: Transportation
- G16Y 20/20: relating to the thing itself
- G16Y 40/10: Detection; Monitoring
- B60W 2420/403: Image sensing, e.g. optical camera
- G05D 2111/14: {Non-visible signals, e.g. IR or UV signals}
- G05D 2111/17: {Coherent light, e.g. laser signals}
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G16Y: INFORMATION AND COMMUNICATION TECHNOLOGY SPECIALLY ADAPTED FOR THE INTERNET OF THINGS [IoT]

## Abstract

There is provided an information processing device including: a first processor that outputs point information obtained by recognizing a captured object as a point from an image of the object captured by a first camera provided in a vehicle; a second processor that outputs identification information for identifying the captured object from an image of the object captured by a second camera which is provided in the vehicle and faces a direction corresponding to a direction of the first camera; and a third processor that associates the point information output from the first processor with the identification information output from the second processor, in which the third processor further performs driving control of the vehicle based on a result obtained by simulating a movement of the object on a digital map on which the point information and the identification information which are associated with each other are plotted.

## Claims

### Claim 1

An information processing system comprising:
circuitry configured to:
output point information obtained by recognizing a captured object as a point from an image of the object captured by a first camera provided in a vehicle;
output identification information for identifying the captured object from an image of the object captured by a second camera which is provided in the vehicle and faces a direction corresponding to a direction of the first camera;
associate the point information with the identification information; and
perform driving control of the vehicle based on a result obtained by simulating a movement of the object on a digital map on which the point information and the ide.

### Claim 2 (depends on 1)

The information processing system according to claim 1, wherein the first camera captures images at a frame rate of 100 frames per second or higher, and the second camera captures images at a frame rate lower than the frame rate of the first camera.

### Claim 3 (depends on 1)

The information processing system according to claim 1, wherein the point information includes position information indicating a position of the object in a three-dimensional orthogonal coordinate system and movement information indicating a movement direction and a movement speed of the object.

### Claim 4 (depends on 1)

The information processing system according to claim 1, wherein the circuitry is configured to associate the point information with the identification information based on a correspondence between position information included in the point information and position information output together with the identification information.

### Claim 5 (depends on 1)

The information processing system according to claim 1, wherein:
the point information includes movement information indicating a movement of the object,
a collision risk between the vehicle and the object is predicted in the simulation by using each movement pattern of the object that is determined according to the movement information and the identification information, and
the circuitry is configured to:
perform driving control of the vehicle so as to avoid a collision in which the collision risk is equal to or higher than a threshold value, and
perform driving control of the vehicle such that traffic congestion of subsequent vehicles does not occur in a case where all collision risks predicted in the simulation are lower than the threshold value.

### Claim 6 (depends on 1)

The information processing system according to claim 1, wherein in the simulation, a collision risk between the vehicle and the object is predicted by using a point representing the object or a polygon surrounding a contour of the object.

### Claim 7 (depends on 1)

The information processing system according to claim 1, further comprising a server communicatively coupled to the circuitry via a network, wherein the server is configured to perform the simulation of the movement of the object on the digital map.

### Claim 8 (depends on 7)

The information processing system according to claim 7, wherein the circuitry is communicatively connected to the server via a gateway, the gateway configured to prevent the circuitry from being directly accessed from outside the vehicle.

### Claim 9 (depends on 1)

The information processing system according to claim 1, wherein the first camera has a variable frame rate, and the circuitry is configured to change the frame rate of the first camera according to a risk score related to an external environment of the vehicle.

### Claim 10 (depends on 1)

The information processing system according to claim 1, wherein the first camera includes a left camera and a right camera, and the circuitry is configured to derive a z coordinate value of the object as the point information based on images captured by the left camera and the right camera using a principle of a stereo camera.

### Claim 11 (depends on 1)

The information processing system according to claim 1, further comprising a radar provided in the vehicle, the radar configured to acquire three-dimensional point cloud data of the object, wherein:
the circuitry is configured to derive coordinate values of the object on three coordinate axes of a three-dimensional orthogonal coordinate system by combining coordinate values derived from images captured by the first camera with a z coordinate value indicated by the three-dimensional point cloud data, and a timing at which the first camera captures images is synchronized with a timing at which the radar acquires the three-dimensional point cloud data.

### Claim 12 (depends on 1)

The information processing system according to claim 1, wherein the first camera includes a visible light camera and an infrared camera, the circuitry is configured to output the point information based on at least one of a visible light image captured by the visible light camera or an infrared image captured by the infrared camera, and a timing at which the visible light camera captures the visible light image is synchronized with a timing at which the infrared camera captures the infrared image.

### Claim 13 (depends on 1)

The information processing system according to claim 1, wherein the first camera includes an event camera configured to capture event images by extracting a difference between an image captured at a current timing and an image captured at a previous timing, and the circuitry is configured to output the point information based on the event images.

### Claim 14 (depends on 1)

The information processing system according to claim 1, further comprising:
a motor assembly including an in-wheel motor provided in each of a plurality of wheels of the vehicle; and
a suspension assembly including a suspension mechanism supporting each of the plurality of wheels,
wherein the circuitry is configured to calculate a plurality of control variables for controlling a wheel speed and an inclination of each of the plurality of wheels and for controlling the suspension mechanism supporting each of the plurality of wheels, and control the motor assembly and the suspension assembly based on the plurality of control variables.

### Claim 15 (depends on 1)

The information processing system according to claim 1, further comprising cooling hardware configured to cool the circuitry using at least one of air cooling, water cooling, or liquid nitrogen cooling, wherein the circuitry is configured to predict an operation status of the circuitry based on a detection result of the object and cause the cooling hardware to cool the circuitry based on the predicted operation status.

### Claim 16 (depends on 1)

The information processing system according to claim 1, wherein the circuitry includes a graphics neural network processing unit and a central processing unit, the graphics neural network processing unit configured to perform processing related to image recognition, and the central processing unit configured to perform processing related to vehicle control.

### Claim 17 (depends on 1)

The information processing system according to claim 1, wherein the circuitry is configured to control an operation of a robot including arms, palms, fingers, and feet based on the point information and the identification information.

### Claim 18

An information processing system comprising:
a first camera provided in a vehicle, the first camera having a variable frame rate;
a second camera provided in the vehicle, the second camera facing a direction corresponding to a direction of the first camera;
a motor assembly including an in-wheel motor provided in each of a plurality of wheels of the vehicle;
a suspension assembly including a suspension mechanism supporting each of the plurality of wheels;
a server communicatively coupled via a network; and
circuitry configured to:
output point information obtained by recognizing a captured object as a point from an image of the object captured by the first camera,
output identification information for identifying the captured object from an image of the object captured by the second camera,
associate the point information with the identification information,
change the frame rate of the first camera according to a risk score related to an external environment of the vehicle,
calculate a plurality of control variables for controlling a wheel speed and an inclination of each of the plurality of wheels and for controlling the suspension mechanism supporting each of the plurality of wheels, and
control the motor assembly and the suspension assembly based on the plurality of control variables and based on a result obtained by simulating, by the server, a movement of the object on a digital map on which the point information and the identification information which are associated with each other are plotted.

### Claim 19 (depends on 18)

The information processing system according to claim 18, wherein the circuitry is configured to obtain the plurality of control variables by inputting, to a deep learning model, an integral value obtained by time-integrating a delta value of a function indicating a behavior of each of a plurality of variables including at least one of air resistance, road resistance, or a slip coefficient, the deep learning model outputting a control variable corresponding to the input integral value.

### Claim 20

An information processing method executed by a computer, the method comprising:
outputting point information obtained by recognizing a captured object as a point from an image of the object captured by a first camera provided in a vehicle;
outputting identification information for identifying the captured object from an image of the object captured by a second camera which is provided in the vehicle and faces a direction corresponding to a direction of the first camera;
associating the point information with the identification information; and
performing driving control of the vehicle based on a result obtained by simulating a movement of the object on a digital map on which the point information and the identification information which are associated with each other are plotted.
