# US 12,112,641 B2 · Multi-vehicle collaboration with group visualization

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,112,641 B2
- Document ID: US12112641B2
- Status: Granted
- Kind: B2
- Application number: 17/672,079
- Filed: 2022-02-15
- Published: 2023-08-17
- Granted: 2024-10-08
- Earliest claimed: 2022-02-15
- Assignees: International Business Machines Corporation
- Applicants: International Business Machines Corporation
- Inventors: Thomas Jefferson Sandridge; Jeremy R. Fox; Tushar Agrawal; Sarbajit K. Rakshit
- Web page: https://patentgrep.com/patent/US12112641B2

## CPC

- G08G 1/22: {Platooning, i.e. convoy of communicating vehicles}
- B60K 35/10: Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
- B60K 35/28: characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
- B60W 30/16: Control of distance between vehicles, e.g. keeping a distance to preceding vehicle
- B60W 40/02: related to ambient conditions
- B60W 50/14: Means for informing the driver, warning the driver or prompting a driver intervention
- G05D 1/0295: {by at least one leading vehicle of the fleet}
- G05D 1/6985: {using a lead vehicle, e.g. primary-secondary arrangements}
- H04W 4/46: for vehicle-to-vehicle communication [V2V]
- B60K 35/213: {Virtual instruments}
- B60K 2360/176: Camera images
- B60K 2360/177: Augmented reality
- B60W 2050/146: {Display means}
- B60W 2556/65: Data transmitted between vehicles
- G08G 1/20: {Monitoring the location of vehicles belonging to a group, e.g. fleet of vehicles, countable or determined number of vehicles}
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})
- B60K: ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72)

## Abstract

A computer-implemented method to provide collaboration between a group of vehicles. The method includes linking two or more vehicles into a group, wherein each vehicle is partially automated, includes a plurality of sensors, and are performing a common task. The method further includes generating, for each vehicle, a 360 degree view of the vehicle with data from the plurality of sensors. The method includes aggregating the 360 degree view for each vehicle into a group view, that includes all the grouped vehicles. The method includes displaying the group view on an augmented reality (AR) device a, where the group view includes a relative location for each vehicle in the group. The method includes applying a set of rules to the group of vehicles, wherein the rules provide instructions to each vehicle how to operate within the group, where the set of rules is configured to complete the task.

## Claims

### Claim 1

A computer-implemented method comprising:
linking, over a network, two or more vehicles into a group of vehicles, wherein each of the two or more vehicles are at least partially automated, each of the vehicles includes a plurality of sensors, and each of the vehicles are performing a common task;
receiving, generating, for each vehicle, a 360 degree view from each vehicle of the two or more vehicles, wherein each 360 view is based on data from the plurality of sensors;
aggregating the 360 degree view for each vehicle into a group view, that includes each vehicle of the two or more vehicles;
sending, to each vehicle, the group view, wherein the group view is configured to be displayed on an augmented reality (AR) device in each vehicle, wherein the group view includes a relative location for each vehicle in the group of vehicles; and
monitoring a set of rules applied to the group of vehicles, wherein the rules provide instructions to each vehicle how to operate within the group, wherein the set of rules is configured to complete the common task.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, further comprising:
adding a new vehicle to the group, wherein the adding comprises:
linking the new vehicle to each of the two or more vehicles;
updating the group view to include the new vehicle; and
applying the set of rules to the new vehicle.

### Claim 3 (depends on 2)

The computer-implemented method of claim 2, wherein the adding is responsive to the new vehicle requesting to be added to the group.

### Claim 4 (depends on 3)

The computer-implemented method of claim 3, further comprising:
changing, in response to the adding the new vehicle to the group, a relative distance between a first vehicle of the two or more vehicles, and a second vehicle of the two or more vehicles;
updating the set of rules to initiate the changing of the relative distance; and
updating the group view to display to updated relative distance.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1, further comprising:
updating the set of rules based on a change in environmental conditions.

### Claim 6 (depends on 1)

The computer-implemented method of claim 1, wherein the group view includes a visual indication of, for each vehicle, a speed, a direction, and distance between each vehicle and each remaining vehicle in the group.

### Claim 7 (depends on 6)

The computer-implemented method of claim 6, wherein the group view includes at least one object not in the group.

### Claim 8 (depends on 7)

The computer-implemented method of claim 7, wherein the at least one object includes a non-group vehicle.

### Claim 9 (depends on 1)

The computer-implemented method of claim 1, further comprising:
updating a configuration of the group of vehicles, wherein the updating comprises:
receiving, by the first vehicle, an input configured to perform the updating of the configuration;
comparing a final position of each vehicle to a current position of each vehicle in the group;
generating, in response to the comparing, a set of actions and an order of action required to perform the updating, wherein each action is correlated to one vehicle of the group; and
executing each action of the set of actions in the order.

### Claim 10 (depends on 9)

The computer-implemented method of claim 9, wherein the executing each action further comprises:
sending each action of the set of actions to the correlated vehicle; and
receiving a notification of completion prior to sending the next action of the set of actions.

### Claim 11

A system comprising:
a processor; and
a computer-readable storage medium communicatively coupled to the processor and storing program instructions which, when executed by the processor, are configured to cause the processor to:
link, over a network, two or more vehicles into a group of vehicles, wherein each of the two or more vehicles are at least partially automated, each of the vehicles includes a plurality of sensors, and each of the vehicles are performing a common task;
generate, for each vehicle, a 360 degree view of each vehicle with data from the plurality of sensors;
aggregate the 360 degree view for each vehicle into a group view, that includes each vehicle of the two or more vehicles;
display the group view on an augmented reality (AR) device a first vehicle, wherein the group view includes a relative location for each vehicle in the group of vehicles; and
apply a set of rules to the group of vehicles, wherein the rules provide instructions to each vehicle how to operate within the group, wherein the set of rules is configured to complete the common task.

### Claim 12 (depends on 11)

The system of claim 11, wherein the program instructions are further configured to cause the processor to:
add a new vehicle to the group of vehicles, wherein the adding includes:
linking the new vehicle to each of the two or more vehicles;
updating the group view to include the new vehicle; and
applying the set of rules to the new vehicle.

### Claim 13 (depends on 12)

The system of claim 12, wherein the addition of the new vehicle is in response to the new vehicle requesting to be added to the group.

### Claim 14 (depends on 11)

The system of claim 11, wherein the group view includes a visual indication of, for each vehicle, a speed, a direction, and distance between each vehicle and each remaining vehicle in the group.

### Claim 15 (depends on 14)

The system of claim 14, wherein the group view includes at least one object not in the group.

### Claim 16

A computer program product, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processing unit to cause the processing unit to:
determine two or more vehicles have linked over a network into a group of vehicles, wherein the group is configured to perform a common task, and each of the two or more vehicles:
are at least partially automated;
includes plurality of sensors; and
vehicles are performing the common task;
receive, from each vehicle, a 360 degree view from each vehicle of the two or more vehicles, wherein each 360 view is based on data from the plurality of sensors;
aggregate the 360 degree view for each vehicle into a group view, that includes each vehicle of the two or more vehicles;
send, to each vehicle, the group view, wherein the group view is configured to be displayed on an augmented reality (AR) device in each vehicle, wherein the group view includes a relative location for each vehicle in the group of vehicles; and
monitor the set of rules applied to the group of vehicles, wherein the rules provide instructions to each vehicle how to operate within the group, wherein the set of rules is configured to complete the common task.

### Claim 17 (depends on 16)

The computer program product of claim 16, wherein the program instructions are further configured to cause the processing unit to:
add a new vehicle to the group view, wherein the adding includes:
determining the new vehicle is linked to each of the two or more vehicles;
updating the group view to include the new vehicle; and
sending the updated view to each vehicle including the new vehicle.

### Claim 18 (depends on 17)

The computer program product of claim 17, wherein the addition of the new vehicle is in response to the new vehicle requesting to be added to the group.

### Claim 19 (depends on 16)

The computer program product of claim 16, wherein the program instructions are further configured to cause the processing unit to:
determine a first vehicle is non-compliant with a first rule; and
send, a first instruction to the first vehicle, wherein the first instruction is configured to be compliant with the first rule, and the first instruction can be executed by the at least partially automated portion of the first vehicle.

### Claim 20 (depends on 19)

The computer program product of claim 19, wherein the program instructions are further configured to cause the processing unit to:
send, in response to determining the first vehicle is non-compliant with the first rule, a second instruction to a second vehicle configured to change a location of the second vehicle in the group to allow the first vehicle to execute the first instruction.