Virtual environment creation for vehicle teleoperation

A method for generating a virtual environment for use in control of a host vehicle comprising:

• generating, via a sensor, sensor data representative of an environment proximate to the host vehicle;
• determining an environmental condition in response to the sensor data, wherein the environmental condition includes a measure of environmental visibility and image saturation;
• determining a network condition in response to a transceiver performance data;
• capturing an image of the environment proximate to the host vehicle;
• generating, by a processor, a modulated data including the image and the sensor data in response to the environmental condition being less than an environmental threshold value and the network condition exceeding a threshold network performance level;
• generating the modulated data without the image in response to at least one of the environmental condition exceeding the environmental threshold value and the network condition not exceeding the threshold network performance level;
• wherein the modulated data without the image comprises the virtual environment including a plurality of bounding boxes representing objects detected in the sensor data;
• wherein a size of each of the plurality of bounding boxes is inversely proportional to a magnitude of the network condition;
• generating a predictive trajectory of the host vehicle for a defined duration of time based on current steering, brake, and throttle inputs and wherein the modulated data includes the predictive trajectory for display on a remote control station alongside the virtual environment and wherein the modulated data includes an object list and wherein at least one of a plurality of objects listed in the object list are represented as one of the plurality of bounding boxes representing a cluster of objects within the environment proximate to the host vehicle and wherein the one of the plurality of bounding boxes representing the cluster of objects is determined in response to at least one of a spatial gap between each of the cluster of objects being less than a width required for the host vehicle to pass through and a host vehicle operational parameter selected from a group consisting of vehicle speed, steering angle, and an optimized field of view for a detected turning maneuver;
• and
• wherein the method further includes clustering a plurality of proximate objects into a single bounding box within the modulated data when the network condition is below a clustering threshold;
• transmitting the modulated data to the remote control station;
• receiving a vehicle control data from the remote control station;
• and
• controlling, via a vehicle controller, the host vehicle in response to the vehicle control data.

A system for generating a virtual environment for use in a vehicle remote teleoperation system for control of a host vehicle comprising:

• an environmental sensor for detecting an environmental condition proximate to the host vehicle, wherein the environmental condition includes a measure of environmental visibility and image saturation;
• a sensor configured to generate a sensor data representative of an environment proximate to the host vehicle;
• a camera configured to capture an image data representative of the environment proximate to the host vehicle;
• a processor configured to:
• determine a network condition;
• generate a modulated data including the image data and the sensor data in response to the environmental condition being less than an environmental threshold value and the network condition exceeding a network threshold condition;
• and
• generate the modulated data without the image data in response to at least one of the environmental condition exceeding the environmental threshold value and the network condition not exceeding the network threshold condition;
• wherein the modulated data without the image data comprises the virtual environment including a plurality of bounding boxes representing objects detected in the sensor data;
• wherein a size of each of the plurality of bounding boxes is inversely proportional to a magnitude of the network condition;
• generate a predictive trajectory of the host vehicle for a defined duration of time based on current steering, brake, and throttle inputs and wherein the modulated data includes the predictive trajectory for display on a remote control station alongside the virtual environment and wherein the modulated data includes an object list and wherein at least one of a plurality of objects listed in the object list are represented as one of the plurality of bounding boxes representing a cluster of objects within the environment proximate to the host vehicle and wherein the one of the plurality of bounding boxes representing the cluster of objects is determined in response to at least one of a spatial gap between each of the cluster of objects being less than a width required for the host vehicle to pass through and a host vehicle operational parameter selected from a group consisting of vehicle speed, steering angle, and an optimized field of view for a detected turning maneuver;
• and
• wherein the processor is further configured to cluster a plurality of proximate objects into a single bounding box within the modulated data when the network condition is below a clustering threshold;
• a transceiver configured to transmit the modulated data to the remote control station and receive a vehicle control data from the remote control station;
• and
• a vehicle controller for controlling the host vehicle in response to the vehicle control data, the vehicle controller configured to automatically transition from a remote teleoperation mode to an onboard advanced driver assistance system (ADAS) mode in response to the network condition falling below a critical safety threshold.