# US 2023/0053340 A1 · Using neural network to optimize operational parameter of vehicle while achieving favorable emotional state of rider

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2023/0053340 A1
- Document ID: US20230053340A1
- Status: Published application
- Kind: A1
- Application number: 17/978,035
- Filed: 2022-10-31
- Published: 2023-02-23
- Earliest claimed: 2018-09-30
- Assignees: Strong Force TP Portfolio 2022, LLC
- Applicants: Strong Force Tp Portfolio 2022, LLC
- Inventors: Charles Howard Cella
- Web page: https://patentgrep.com/patent/US20230053340A1

## CPC

- G06Q 30/0265: {Vehicular advertisement}
- A61B 5/165: {Evaluating the state of mind, e.g. depression, anxiety}
- A61B 5/168: {Evaluating attention deficit, hyperactivity}
- A61B 5/6893: {Cars}
- A61B 5/7264: {Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems}
- B60W 40/08: related to drivers or passengers
- G01C 21/3438: {Rendezvous; Ride sharing}
- G01C 21/3461: {Preferred or disfavoured areas, e.g. dangerous zones, toll or emission zones, intersections, manoeuvre types or segments such as motorways, toll roads or ferries}
- G01C 21/3469: {Fuel consumption; Energy use; Emission aspects}
- G01C 21/3617: {using user history, behaviour, conditions or preferences, e.g. predicted or inferred from previous use or current movement}
- G05B 13/027: {using neural networks only}
- G05D 1/0022: {characterised by the communication link (data switching networks in general H04L12/00)}
- G05D 1/0088: {characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours (using knowledge based models G06N5/00)}
- G05D 1/0212: {with means for defining a desired trajectory (involving a plurality of land vehicles G05D1/0287)}
- G05D 1/0287: {involving a plurality of land vehicles, e.g. fleet or convoy travelling (traffic control systems for road vehicles G08G1/00, particularly anticollision systems G08G1/16)}
- G05D 1/20: Control system inputs
- G05D 1/224: Output arrangements on the remote controller, e.g. displays, haptics or speakers
- G05D 1/225: operated by off-board computers
- G05D 1/226: Communication links with the remote-control arrangements
- G05D 1/227: Handing over between remote control and on-board control; Handing over between remote control arrangements
- G05D 1/228: Command input arrangements located on-board unmanned vehicles
- G05D 1/229: Command input data, e.g. waypoints
- G05D 1/24: Arrangements for determining position or orientation
- G05D 1/43: Control of position or course in two dimensions [2D]
- G05D 1/646: Following a predefined trajectory, e.g. a line marked on the floor or a flight path
- G05D 1/69: Coordinated control of the position or course of two or more vehicles
- G05D 1/692: involving a plurality of disparate vehicles
- G05D 1/81: Handing over between on-board automatic and on-board manual control
- G06F 18/2414: {Smoothing the distance, e.g. radial basis function networks [RBFN]}
- G06F 40/40: Processing or translation of natural language (natural language analysis G06F40/20; semantic analysis G06F40/30)
- G06N 3/0418: {using chaos or fractal principles}
- G06N 3/044: Recurrent networks, e.g. Hopfield networks
- G06N 3/0442: characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
- G06N 3/045: Combinations of networks
- G06N 3/0464: Convolutional networks [CNN, ConvNet]
- G06N 3/08: Learning methods
- G06N 3/082: modifying the architecture, e.g. adding, deleting or silencing nodes or connections
- G06N 3/086: using evolutionary algorithms, e.g. genetic algorithms or genetic programming
- G06N 3/09: Supervised learning
- G06N 3/092: Reinforcement learning
- G06N 3/126: Evolutionary algorithms, e.g. genetic algorithms or genetic programming
- G06N 20/00: Machine learning
- G06Q 30/0208: Trade or exchange of goods or services in exchange for incentives or rewards
- G06Q 50/188: {Electronic negotiation}
- G06Q 50/40: Business processes related to the transportation industry (shipping G06Q10/083)
- G06V 10/764: using classification, e.g. of video objects
- G06V 10/82: using neural networks
- G06V 20/56: exterior to a vehicle by using sensors mounted on the vehicle
- G06V 20/59: inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V 20/597: {Recognising the driver's state or behaviour, e.g. attention or drowsiness}
- G06V 20/64: Three-dimensional [3D] objects
- G07C 5/006: {Indicating maintenance}
- G07C 5/008: {communicating information to a remotely located station (transmission systems for measured values G08C)}
- G07C 5/02: Registering or indicating driving, working, idle, or waiting time only (apparatus forming part of taximeters G07B13/00)
- G07C 5/08: Registering or indicating performance data other than driving, working, idle, or waiting time, with or without registering driving, working, idle or waiting time
- G07C 5/0808: {Diagnosing performance data (testing of vehicles G01M17/00; testing of electrical installation on vehicles G01R31/005)}
- G07C 5/0816: {Indicating performance data, e.g. occurrence of a malfunction}
- G07C 5/0866: {the electronic data carrier being a digital video recorder in combination with video camera}
- G07C 5/0891: {Video recorder in combination with video camera}
- G10L 15/16: using artificial neural networks
- G10L 25/63: for estimating an emotional state
- A61B 5/0059: {using light, e.g. diagnosis by transillumination, diascopy, fluorescence (photoacoustic A61B5/0093; optical measurement of heart rate A61B5/02416; optical measurement of blood flow A61B5/0261; optical measurement of analytes A61B5/1455)}
- A61B 5/0533: Measuring galvanic skin response
- A61B 5/1176: {Recognition of faces}
- A61B 5/163: {by tracking eye movement, gaze, or pupil change}
- A61B 5/369: Electroencephalography [EEG]
- A61B 2503/22: Motor vehicles operators, e.g. drivers, pilots, captains
- A61B 2562/0219: Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches
- A61B 2562/0223: Magnetic field sensors
- B60W 2040/0881: {Seat occupation; Driver or passenger presence}
- G01C 21/3484: {Personalized, e.g. from learned user behaviour or user-defined profiles}
- G05D 2101/10: using artificial intelligence [AI] techniques
- G06N 3/02: Neural networks
- G06N 3/048: Activation functions
- G06Q 10/40: Business processes related to social networking or social networking services
- G06Q 10/42: Determination of affinities or common interests between users
- G06Q 10/44: Identification of trends within social networks, e.g. identification of trending topics
- G06Q 30/0281: {Customer communication at a business location, e.g. providing product or service information, consulting}
- G08G 1/0968: Systems involving transmission of navigation instructions to the vehicle
- G08G 1/20: {Monitoring the location of vehicles belonging to a group, e.g. fleet of vehicles, countable or determined number of vehicles}
- Y02T 10/62: Hybrid vehicles
- Y02T 10/70: Energy storage systems for electromobility, e.g. batteries
- Y02T 10/7072: Electromobility specific charging systems or methods for batteries, ultracapacitors, supercapacitors or double-layer capacitors
- Y02T 90/12: Electric charging stations
- G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- A61B: DIAGNOSIS; SURGERY; IDENTIFICATION (usefulness limited to only animals A61D)
- B60W: CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S)
- G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G)
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06N: COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G07C: TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G08G: TRAFFIC CONTROL SYSTEMS (guiding railway traffic, ensuring the safety of railway traffic B61L; arrangement of road signs or traffic signals E01F9/00; radar or analogous systems, sonar systems, lidar systems specially adapted for traffic control G01S13/91, G01S15/88, G01S17/88; {radar or analogous systems, sonar systems, lidar systems specially adapted for anti-collision purposes G01S13/93, G01S15/93, G01S17/93})
- Y02T: CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION

## Abstract

A system for transportation includes a vehicle, a rider occupying the vehicle, and a hybrid neural network. The hybrid neural network includes a first neural network to process a sensor input corresponding to the rider to determine an emotional state of the rider, and a second neural network to optimize at least one operating parameter of the vehicle to improve the emotional state of the rider.

## Claims

### Claim 1

A system for transportation, comprising:
a vehicle occupied by a rider; and
a hybrid neural network, including: a first neural network to process a sensor input corresponding to the rider to determine an emotional state of the rider; and a second neural network to optimize at least one operating parameter of the vehicle to improve the emotional state of the rider.

### Claim 2 (depends on 1)

The system for transportation of claim 1 wherein the at least one operating parameter of the vehicle that is optimized affects: a route of the vehicle, a speed of the vehicle, an acceleration of the vehicle, a deceleration of the vehicle, and a proximity to other vehicles along the route.

### Claim 3 (depends on 1)

The system for transportation of claim 1 wherein a sensor that produces the sensor input corresponding to the rider is to gather physiological sensed data indicative of a hormone level of the rider, wherein the physiological sensed data includes galvanic skin response indicative of a cortisol level of the rider.

### Claim 4 (depends on 3)

The system for transportation of claim 3 wherein the first neural network is to determine the emotional state of the rider through analysis of the physiological sensed data.

### Claim 5 (depends on 3)

The system for transportation of claim 3 wherein the sensor is deployed in the vehicle.

### Claim 6 (depends on 3)

The system for transportation of claim 3 wherein the sensor is a wearable sensor.

### Claim 7 (depends on 1)

The system for transportation of claim 1 wherein the second neural network is to optimize the at least one operating parameter based on a correlation between a vehicle operating state and the emotional state of the rider.

### Claim 8 (depends on 1)

The system for transportation of claim 1 wherein the hybrid neural network further includes:
a third neural network to classify social media data sourced from a plurality of social media sources as indicative of an effect on the system for transportation;
a fourth neural network to predict at least one aspect of rider satisfaction affected by an effect on the system for transportation derived from the social media data classified as indicative of the effect on the system for transportation; and
a fifth neural network to optimize the at least one aspect of rider satisfaction for at least one rider occupying the vehicle.

### Claim 9 (depends on 8)

The system for transportation of claim 8 wherein at least one of the neural networks in the hybrid neural network is a convolutional neural network.

### Claim 10 (depends on 8)

The system for transportation of claim 8 wherein the at least one aspect of rider satisfaction is optimized by: predicting an entertainment option for presenting to the rider; or optimizing route planning for the vehicle occupied by the rider.

### Claim 11 (depends on 8)

The system for transportation of claim 8 wherein the at least one aspect of rider satisfaction is a rider state and optimizing the at least one aspect of rider satisfaction comprises optimizing the rider state.

### Claim 12 (depends on 8)

The system for transportation of claim 8 wherein social media data specific to the rider is analyzed to determine at least one optimizing action likely to optimize the at least one aspect of rider satisfaction.

### Claim 13 (depends on 12)

The system for transportation of claim 12 wherein the at least one optimizing action is one of adjusting a routing plan to include passing points of interest to the rider, deriving an altruistic benefit, or presenting entertainment options.

### Claim 14 (depends on 13)

The system for transportation of claim 13 wherein the altruistic benefit is reduction of environmental impact.

### Claim 15 (depends on 8)

The system for transportation of claim 8 wherein the social media data includes at least one of:
social media posts; social media feeds; like or dislike activity detected in the social media; indications of relationships; user behavior; discussion threads; chats; or an indication of a celebrity at a location.

### Claim 16 (depends on 8)

The system for transportation of claim 8 wherein the social media data includes at least one of presence of a rare phenomenon at a location; entertainment options; risk-related conditions; or modes of transportation used with an event.

### Claim 17 (depends on 8)

The system for transportation of claim 8 wherein the effect on the system for transportation is one of a reduced carbon footprint, or reduced pollution.

### Claim 18 (depends on 8)

The system for transportation of claim 8 wherein the optimized at least one aspect of rider satisfaction is one of an operating state of the vehicle, an in-vehicle state, a rider state, a routing state, or a user experience state.

### Claim 19 (depends on 8)

The system for transportation of claim 8 wherein a characterization of an outcome of the optimizing in the social media data is used as feedback to improve the optimizing.

### Claim 20 (depends on 19)

The system for transportation of claim 19 wherein the feedback includes at least one of likes or dislikes of the outcome; social media activity referencing the outcome; trending of the social media activity referencing the outcome; hash tags associated with the outcome; ratings of the outcome; or requests for the outcome.
