# US 12,005,918 B2 · System and method for providing assistance to vehicle occupants > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,005,918 B2 - Document ID: US12005918B2 - Status: Granted - Kind: B2 - Application number: 17/456,170 - Filed: 2021-11-23 - Published: 2023-03-02 - Granted: 2024-06-11 - Earliest claimed: 2021-09-01 - Assignees: Wipro Limited - Applicants: Wipro Limited - Inventors: Nilanjan Biswas; Shantanu Banerjee; Kyunglack Kim; Abhishek Chatterjee - Web page: https://patentgrep.com/patent/US12005918B2 ## CPC - B60W 50/14: Means for informing the driver, warning the driver or prompting a driver intervention - B60K 35/10: Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor - B60K 35/28: characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver - B60K 35/85: Arrangements for transferring vehicle- or driver-related data - B60W 40/04: Traffic conditions - B60W 40/08: related to drivers or passengers - G06N 3/08: Learning methods - B60K 35/25: using haptic output - B60K 35/26: using acoustic output - B60K 2360/5915: Inter vehicle communication - B60K 2360/592: Data transfer involving external databases - B60W 2554/4029: Pedestrians - B60W 2555/20: Ambient conditions, e.g. wind or rain - B60W 2556/45: External transmission of data to or from the vehicle - G06F 3/0481: based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance - G06N 3/04: Architecture, e.g. interconnection topology - G06N 5/04: Inference or reasoning models - B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT - B60K: ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES - G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A system and method for providing assistance to vehicle occupants is disclosed. The method includes the steps of receiving a set of static information and a set of dynamic information associated with each of a vehicle, a set of Human Machine Interfaces (HMIs), and an occupant within the vehicle; processing the set of static information and the set of dynamic information through at least one of a first Artificial Neural Network (ANN) Model and a rule engine; determining an accessibility score for the occupant in response to an event based on an output of the processing; generating at least one of at least one accessibility compliant HMI content for at least one of the set of HMIs and vehicle control content for the vehicle based on the accessibility score; and executing at least one of the at least one accessibility compliant HMI content and the vehicle control content. ## Claims ### Claim 1 A method of providing assistance to vehicle occupants, the method comprising: receiving, by an assistance device, a set of static information and a set of dynamic information associated with each of a vehicle, a set of Human Machine Interfaces (HMIs), and at least one occupant within the vehicle; processing, by the assistance device, the set of static information and the set of dynamic information through at least one of a first Artificial Neural Network (ANN) Model and a rule engine by: determining, by the assistance device, an environmental context from vehicle data and environmental data provided in at least one of the set of static information and the set of dynamic information, wherein the environmental data includes information associated with at least one of current weather situation, traffic, drive terrain, ambient light, nearby vehicles, or pedestrians; determining, by the assistance device, an accessibility context from at least one of the set of static information and the set of dynamic information, wherein the accessibility context is determined for at least one occupant for a particular time instance based on information, provided in at least one of, the set of static information and in the set of dynamic information, including the vehicle data, health information of the at least one occupant, the environmental data, driving preferences of the at least one occupant, HMI state information, and pre-determined accessibility data pertaining to the at least one occupant; determining, by the assistance device, an accessibility score for the occupant in response to an event based on the determined environmental context and the determined accessibility context; generating, by the assistance device, at least one of at least one accessibility compliant HMI content for at least one of the set of HMIs and vehicle control content for the vehicle based on the accessibility score, wherein the vehicle control content is associated with driving instructions; and executing, by the assistance device, at least one of the at least one accessibility compliant HMI content and the vehicle control content, wherein executing the vehicle control content includes automatically performing an action to control the vehicle. ### Claim 2 (depends on 1) The method of claim 1, further comprising: processing the set of static information, the set of dynamic information, and a pattern data associated with the occupant through at least one of a second ANN Model and a cloud-based rule engine; determining a cloud-based accessibility score for the occupant in response to the event based on an output of the processing through at least one of the second ANN Model and the cloud-based rule engine; generating at least one of at least one cloud-based accessibility compliant HMI content for at least one of the set of HMIs for the occupant and cloud-based vehicle control content for the vehicle, based on the cloud-based accessibility score; validating of each of the at least one accessibility compliant HMI content, the vehicle control content, the at least one cloud-based accessibility compliant HMI content, and the cloud-based vehicle control content, based on a current context associated with the occupant and the vehicle; and rendering, in response to the validating, one of: at least one of the at least one accessibility compliant HMI content and the vehicle control content; or at least one of the at least one cloud-based accessibility compliant HMI content and the cloud-based vehicle control content. ### Claim 3 (depends on 1) The method of claim 1, further comprising: assigning a relative priority to each of the at least one accessibility compliant HMI content and the vehicle control content to generate a priority list, wherein the priority list is dynamically modified based on the event at a time instance; and executing at least one of the at least one accessibility compliant HMI content and the vehicle control content in an order of assigned priority for the time instance. ### Claim 4 (depends on 1) The method of claim 1, further comprising: receiving validation data corresponding to the at least one accessibility compliant HMI content and the vehicle control content from at least one neighbouring vehicle via V2X communication protocol, wherein the validation data corresponds to the set of static information, the set of dynamic information, and the event; and validating the at least one accessibility compliant HMI content and the vehicle control content based on the received validation data. ### Claim 5 (depends on 1) The method of claim 1, wherein generating the at least one accessibility compliant HMI content on at least one of the set of HMIs comprises at least one of: creating a new accessibility compliant HMI content; and modifying an existing accessibility compliant HMI content to generate a modified accessibility compliant HMI content using a plurality of available HMI components, wherein the plurality of available HMI components comprises a vehicle native HMI software, cached HMI content from an associated cloud storage for a similar accessibility score, and cached HMI content from a neighboring vehicle. ### Claim 6 (depends on 1) The method of claim 1, further comprising rendering at least one of the at least one accessibility compliant HMI content and the vehicle control content to at least one of a nearby vehicle and a pedestrian smartphone. ### Claim 7 A system for providing assistance to vehicle occupants, the system comprising: a processor; and a memory communicatively coupled to the processor, wherein the memory stores processor instructions, which, on execution, cause the processor to: receive a set of static information and a set of dynamic information associated with each of a vehicle, a set of Human Machine Interfaces (HMIs), and at least one occupant within the vehicle; process the set of static information and the set of dynamic information through at least one of a first Artificial Neural Network (ANN) Model and a rule engine by: determining an environmental context from vehicle data and environmental data provided in at least one of, the set of static information and the set of dynamic information, wherein the environmental data includes information associated with at least one of current weather situation, traffic, drive terrain, ambient light, nearby vehicles, or pedestrians; determining an accessibility context from at least one of the set of static information and the set of dynamic information, wherein the accessibility context is determined for at least one occupant for a particular time instance based on information, provided in at least one of the set of static information and in the set of dynamic information, including the vehicle data, health information of the at least one occupant, the environmental data, driving preferences of the at least one occupant, HMI state information, and pre-determined accessibility data pertaining to the at least one occupant; determine an accessibility score for the occupant in response to an event based on the determined environmental context and the determined accessibility context; generate at least one of at least one accessibility compliant HMI content for at least one of the set of HMIs and vehicle control content for the vehicle based on the accessibility score, wherein the vehicle control content is associated with driving instructions; and execute at least one of the at least one accessibility compliant HMI content and the vehicle control content, wherein executing the vehicle control content includes automatically performing an action to control the vehicle. ### Claim 8 (depends on 7) The system of claim 7, wherein the processor instructions further cause the processor to: process the set of static information, the set of dynamic information, and a pattern data associated with the occupant through at least one of a second ANN Model and a cloud-based rule engine; determine a cloud-based accessibility score for the occupant in response to the event based on an output of the processing through at least one of the second ANN Model and the cloud-based rule engine; generate at least one of at least one cloud-based accessibility compliant HMI content for at least one of the set of HMIs for the occupant and cloud-based vehicle control content for the vehicle, based on the cloud-based accessibility score; validate contextual appropriateness of each of the at least one accessibility compliant HMI content, the vehicle control content, the at least one cloud-based accessibility compliant HMI content, and the cloud-based vehicle control content, based on a current context associated with the occupant and the vehicle; and render, in response to the validating, one of: at least one of the at least one accessibility compliant HMI content and the vehicle control content; or at least one of the at least one cloud-based accessibility compliant HMI content and the cloud-based vehicle control content. ### Claim 9 (depends on 7) The system of claim 7, wherein the processor instructions further cause the processor to: assign a relative priority to each of the at least one accessibility compliant HMI content and the vehicle control content to generate a priority list, wherein the priority list is dynamically modified based on the event at a time instance; and execute at least one of the at least one accessibility compliant HMI content and the vehicle control content in an order of assigned priority for the time instance. ### Claim 10 (depends on 7) The system of claim 7, wherein the processor instructions further cause the processor to: receive validation data corresponding to the at least one accessibility compliant HMI content and the vehicle control content from at least one neighbouring vehicle via V2X communication protocol, wherein the validation data corresponds to the set of static information, the set of dynamic information, and the event; and validate the at least one accessibility compliant HMI content and the vehicle control content based on the received validation data. ### Claim 11 (depends on 7) The system of claim 7, wherein the processor instructions cause the processor to generate the at least one accessibility compliant HMI content on at least one of the set of HMIs comprises at least one of: create a new accessibility compliant HMI content; and modify an existing accessibility compliant HMI content to generate a modified accessibility compliant HMI content using a plurality of available HMI components, wherein the plurality of available HMI components comprises a vehicle native HMI software, cached HMI content from an associated cloud storage for a similar accessibility score, and cached HMI content from a neighboring vehicle. ### Claim 12 (depends on 7) The system of claim 7, wherein the processor instructions further cause the processor to render at least one of the at least one accessibility compliant HMI content and the vehicle control content to at least one of a nearby vehicle and a pedestrian smartphone. ### Claim 13 A non-transitory computer-readable medium for providing assistance to vehicle occupants, having stored thereon, a set of computer-executable instructions causing one or more processors to perform steps comprising: receiving a set of static information and a set of dynamic information associated with each of a vehicle, a set of Human Machine Interfaces (HMIs), and at least one occupant within the vehicle; processing the set of static information and the set of dynamic information through at least one of a first Artificial Neural Network (ANN) Model and a rule engine by; determining an environmental context from vehicle data and environmental data provided in at least one of, the set of static information and the set of dynamic information, wherein the environmental data includes information associated with at least one of current weather situation, traffic, drive terrain, ambient light, nearby vehicles, or pedestrians; determining an accessibility context from at least one of the set of static information and the set of dynamic information, wherein the accessibility context is determined for at least one occupant for a particular time instance based on information, provided in at least one of, the set of static information and in the set of dynamic information, including the vehicle data, health information of the at least one occupant, the environmental data, driving preferences of the at least one occupant, HMI state information, and pre-determined accessibility data pertaining to the at least one occupant; determining an accessibility score for the occupant in response to an event based on the determined environmental context and the determined accessibility context; generating at least one of at least one accessibility compliant HMI content for at least one of the set of HMIs and vehicle control content for the vehicle based on the accessibility score, wherein the vehicle control content is associated with driving instructions; and executing at least one of the at least one accessibility compliant HMI content and the vehicle control content, wherein executing the vehicle control content includes automatically performing an action to control the vehicle. ### Claim 14 (depends on 13) A non-transitory computer-readable medium of claim 13, wherein the set of computer-executable instructions further causes the one or more processors to perform steps comprising: processing the set of static information, the set of dynamic information, and a pattern data associated with the occupant through at least one of a second ANN Model and a cloud-based rule engine; determining a cloud-based accessibility score for the occupant in response to the event based on an output of the processing through at least one of the second ANN Model and the cloud-based rule engine; generating at least one of at least one cloud-based accessibility compliant HMI content for at least one of the set of HMIs for the occupant and cloud-based vehicle control content for the vehicle, based on the cloud-based accessibility score; validating contextual appropriateness of each of the at least one accessibility compliant HMI content, the vehicle control content, the at least one cloud-based accessibility compliant HMI content, and the cloud-based vehicle control content, based on a current context associated with the occupant and the vehicle; and rendering, in response to the validating, one of: at least one of the at least one accessibility compliant HMI content and the vehicle control content; or at least one of the at least one cloud-based accessibility compliant HMI content and the cloud-based vehicle control content.