# US 2026/0249866 A1 · System and Method for Processing Vehicle Context Information

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2026/0249866 A1
- Document ID: US20260249866A1
- Status: Published application
- Kind: A1
- Application number: 19/549,524
- Filed: 2026-02-25
- Published: 2026-08-27
- Earliest claimed: 2025-02-25
- Assignees: Aptiv Technologies AG
- Applicants: Aptiv Technologies AG
- Inventors: Alexander Barth; Ting-Chieh Yu; Darshil Jariwala; Bind Sunil; Guruprasanna Marimuthu; Arnab Chandra; Gaurav Bansal; Mansi Dhingra
- Web page: https://patentgrep.com/patent/US20260249866A1

## CPC

- B60W 50/08: Interaction between the driver and the control system
- G10L 15/22: Procedures used during a speech recognition process, e.g. man-machine dialogue
- B60K 35/10: Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
- B60K 35/28: characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
- B60K 35/80: Arrangements for controlling instruments
- B60K 35/85: Arrangements for transferring vehicle- or driver-related data
- B60W 50/10: Interpretation of driver requests or demands
- B60W 50/14: Means for informing the driver, warning the driver or prompting a driver intervention
- B60K 2360/146: Instrument input by gesture
- B60K 2360/148: Instrument input by voice
- B60W 2050/143: {Alarm means}
- B60W 2050/146: {Display means}
- B60W 2420/403: Image sensing, e.g. optical camera
- B60W 2556/45: External transmission of data to or from the vehicle
- G06N 20/00: Machine learning
- G10L 13/00: Speech synthesis; Text to speech systems
- G10L 2015/223: {Execution procedure of a spoken command}
- G10L 2015/228: {of application context}
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- B60K: ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS

## Abstract

A technique of processing a command of an occupant of a vehicle is presented in which the command, captured at the vehicle, is serviced by a remote content generation module receiving a multimodal prompt containing the command and information relating to the context of the command. The context is determined using information from vehicle sensors, and the content generation module may be a large language model, able to provide an intuitive response to the command, taking into account the information from the vehicle sensors. A variety of functions are caused to be performed in dependence on the response which is received.

## Claims

### Claim 1

A system for processing a command of an occupant of a vehicle, the system comprising:
a processing module configured to:
receive the command from one of more first sensors, and
determine context of the command using data from one or more second sensors;
a communication module configured to interact with a content generation module; and
an output module, wherein:
the processing module is configured to:
generate a multimodal prompt for a model of the content generation module based on the command and the determined context;
transmit the multimodal prompt to the content generation module via the communication module; and
receive a response from the content generation module via the communication module; and
the multimodal prompt has a plurality of modes,
one of the plurality of modes is text, based on the command,
one of the plurality of modes includes image data, video data, or audio data, based on the data from the one or more second sensors corresponding to the context of the command, and
the output module is configured to cause a task to be performed, based on generated content of the response.

### Claim 2 (depends on 1)

The system of claim 1, wherein:
the output module includes at least one of: a display screen, a head-up display, an augmented reality display, a speaker, and a transmitter for transmitting information to a device; and
the task includes at least one of changing vehicle configuration settings, performing an automated driving assistance function, outputting audio content, outputting visual content, storing information, and communicating with a remote computing system.

### Claim 3 (depends on 1)

The system of claim 1, wherein the processing module is configured to at least one of:
remove or obscure personal information from at least one of the image data, video data, and audio data; or
encrypt the multimodal prompt.

### Claim 4 (depends on 3)

The system of claim 3, wherein the processing module is configured to:
crop image data from data received from the one or more second sensors based on the command, and
include the cropped image data in the multimodal prompt.

### Claim 5 (depends on 1)

The system of claim 1, wherein the processing module includes an ingestion module configured to:
receive information from the one or more second sensors; and
determine contextual information from the one or more second sensors.

### Claim 6 (depends on 5)

The system of claim 5, wherein the processing module is configured to:
access one or more additional information sources defining at least one of location data or points of interest; and
determine contextual information using the additional information in combination with the information from the one or more second sensors.

### Claim 7 (depends on 6)

The system of claim 6, further comprising:
a buffer configured to store images captured by a camera system over a period of time,
wherein the processing module is configured to use the stored images in combination with the additional information to determine contextual information.

### Claim 8 (depends on 1)

The system of claim 1, wherein the processing module is configured to:
determine the command from at least one of a microphone or a camera and to append a timestamp to the command; and
select contextual information from the one or more second sensors that is synchronized with the timestamps for the command.

### Claim 9 (depends on 1)

The system of claim 1, wherein the output module includes a text-to-speech module configured to convert text from the content generation module to a speech output.

### Claim 10 (depends on 1)

The system of claim 1, wherein the communication module is configured to transmit the multimodal prompt to a remote computing system hosting the content generation module.

### Claim 11 (depends on 1)

The system of claim 1, further comprising:
the content generation module,
wherein the content generation module is a first large language model.

### Claim 12 (depends on 11)

The system of claim 11, wherein:
the communication module is configured to transmit the multimodal prompt to a remote computing system hosting a second large language model in response to the processing module determining that a complexity of the multimodal prompt exceeds a threshold level of complexity; and
the processing module is configured to determine the complexity of the multimodal prompt in dependence on processing resources required to determine a response to the multimodal prompt.

### Claim 13 (depends on 1)

The system of claim 1, further comprising a remote computing system hosting the content generation module, wherein:
the content generation module is a large language model,
the large language model is configured to request the communication module to transmit contextual information to the content generation module periodically, and
the processing module is configured to append a timestamp to the transmitted contextual information.

### Claim 14 (depends on 13)

The system of claim 13, further comprising the one or more first sensors and the one or more second sensors, wherein:
the one or more first sensors include at least one of a camera or a microphone configured to at least one of receive a command from an occupant of the vehicle or sense vehicle status information; and
the one or more second sensors are configured to sense at least one of an interior or exterior of the vehicle to determine information relating to at least one of the context of the vehicle or the occupant of the vehicle.

### Claim 15

A computer-implemented method of processing a command of an occupant of a vehicle, comprising:
receiving a command from one of more first sensors,
determining context of the command, using data from one or more second sensors corresponding to the determined context;
generating a multimodal prompt for a model of a content generation module, based on the command and the determined context, wherein a mode of the multimodal prompt is text, based on the command, and a mode of the multimodal prompt includes image data, video data or audio data, based on identified data from the one or more second sensors corresponding to the context of the command;
transmitting the multimodal prompt to the content generation module;
receiving a response from the content generation module; and
causing a task to be performed, based on generated content of the response.
