Generation of grammar-compliant programming language code using machine learning

A system comprising:

• a memory to store a generative language model;
• and
• at least one processing unit to:
• generate a plurality of values using the generative language model, each of the values indicative of a probability of a respective token being a next token of a token sequence generated by the generative language model;
• and
• apply a mask to the plurality of values, the mask operating on each value that corresponds to a token not compliant with a grammar of the programming language to reduce or zero the probability of the token being the next token;
• wherein the generative language model is to determine the next token based on the plurality of values after the mask is applied.

One or more non-transitory computer-readable storage media having stored thereon computer-executable instructions that, when executed by at least one processing unit, cause the at least one processing unit to perform operations comprising:

• generating a plurality of values using a generative language model, each of the values indicative of a probability of a respective token being a next token of a token sequence generated by the generative language model;
• and
• applying a mask to the plurality of values, the mask operating on each value that corresponds to a token not compliant with a grammar of the programming language to reduce or zero the probability of the token being the next token;
• wherein the generative language model determines the next token based on the plurality of values after the mask is applied.