Zero-shot language solver fueled by visual imagination
AssigneeTencent America LLC
InventorsWenlin YAO, Hongming ZHANG, Xiaoyang WANG, Dong YU, Jianshu CHEN, Yue YANG
Life of this patent
Estimating the term…
Abstract
A language processing method performed by at least one processor includes receiving a task input stream and a solution input stream; selecting one of the task input stream and the solution input stream, and providing the selected stream to an image conversion model; creating, based on the selected input stream, a model ensemble of the image conversion model and the language model; and outputting a prediction based on the model ensemble.
Claims20
A language processing method performed by at least one processor, the language method comprising:
- receiving a task input stream and a solution input stream;
- selecting one of the task input stream and the solution input stream, and providing the selected stream to an image conversion model;
- creating, based on the selected input stream, a model ensemble of the image conversion model and the language model;
- and
- outputting a prediction based on the model ensemble.
A language processing apparatus comprising:
- at least one memory configured to store program code;
- and
- at least one processor configured to read the program code and operate as instructed by the program code, the program code comprising:
- receiving code configured to cause the at least one processor to receive a task input stream and a solution input stream;
- selecting code configured to cause the at least one processor to select one of the task input stream and the solution input stream;
- providing code configured to cause the at least one processor to provide the selected input stream to an image conversion model and a language model;
- ensembling code configured to cause the at least one processor to create, based on the selected input stream, a model ensemble of the image conversion model and the language model;
- and
- outputting code configured to cause the at least one processor to output a prediction based on the model ensemble.
A non-transitory computer readable medium having instructions stored therein, which when executed by a processor cause the processor to at least:
- receive a task input stream and a solution input stream;
- select one of the task input stream and the solution input stream;
- provide the selected input stream to an image conversion model and a language model;
- create, based on the selected input stream, a model ensemble of the image conversion model and the language model;
- and
- output a prediction based on the model ensemble.
Dependencies are inferred from the claim text.