System and method of captions for triggers
AssigneesToyota Research Institute, Inc., Toyota Jidosha Kabushiki Kaisha
InventorsGuy Rosman, Yen-Ling Kuo, Stephen G. Mcgill, Simon A.I. Stent
Life of this patent
Estimating the term…
Abstract
A method for triggering capture of diverse driving data from captions is described. The method includes training a discriminator network to identify similarities between a received text description and a received scene description. The method also includes feeding a trained discriminator network with real scene information along with text/sentence descriptions to verify whether the real scene information matches the text/sentence description. The method further includes generating a dataset of diverse driving scenarios retrieved from a dataset of vehicle driving log data in response to a text/sentence query.
Claims18
A method for triggering capture of diverse driving data from captions, the method comprising:
- training a discriminator network to identify similarities between a received text description and a received scene description;
- feeding a trained discriminator network with real scene information along with text/sentence descriptions to form a learned comparison block when the real scene information matches a text/sentence description by:
- encoding the text/sentence descriptions into word tokens, and
- feeding the word tokens into a map encoder and a trajectory encoder for each selected driving scenario from the real scene information to provide a respective scenario signature;
- and
- generating a dataset of diverse driving scenarios retrieved from a dataset of vehicle driving log data in response to an encoded text/sentence query and a matching scenario signature according to the learned comparison block.
A non-transitory computer-readable medium having program code recorded thereon for triggering capture of diverse driving data from captions, the program code being executed by a processor and comprising:
- program code to train a discriminator network to identify similarities between a received text description and a received scene description;
- program code to feed a trained discriminator network with real scene information along with text/sentence descriptions to form a learned comparison block when the real scene information matches a text/sentence description by:
- encoding the text/sentence descriptions into word tokens, and
- feeding the word tokens into a map encoder and a trajectory encoder for each selected driving scenario from the real scene information to provide a respective scenario signature;
- and
- program code to generate a dataset of diverse driving scenarios retrieved from a dataset of vehicle driving log data in response to an encoded text/sentence query and a matching scenario signature according to the learned comparison block.
A system for triggering capture of diverse driving data from captions, the system comprising a processor to execute:
- a discriminator network training module to train a discriminator network to identify similarities between a received text description and a received scene description;
- a discriminator network verification module to feed a trained discriminator network with real scene information along with text/sentence descriptions to form a learned comparison block when the real scene information matches a text/sentence description;
- by:
- encoding the text/sentence descriptions into word tokens, and
- feeding the word tokens into a map encoder and a trajectory encoder for each selected driving scenario from the real scene information to provide a respective scenario signature;
- and
- a dataset generation module to generate a dataset of diverse driving scenarios retrieved from a dataset of vehicle driving log data in response to an encoded text/sentence query and a matching scenario signature according to the learned comparison block.
Dependencies are inferred from the claim text.