Configuration-preserving preprocessor

A system comprising:

• one or more processors;
• and
• one or more computer storage media storing one or more modules of computer program instructions executed by the one or more processors, the one or more modules of computer program instructions comprising:
• a configuration-preserving preprocessor module to:
• determine that a first static conditional in a macro invocation includes one or more nested static conditionals within the branch code associated with the first static conditional;
• and
• hoist each of the one or more nested static conditionals around the macro invocation to a beginning of the branch code associated with the first static conditional, wherein each innermost branch code does not contain a static conditional, and wherein each possible configuration is preserved;
• a parser configured to:
• parse a plurality of tokens, wherein the plurality of tokens comprise language tokens and conditional tokens, wherein various configurations are possible based upon the conditional tokens, wherein the parser comprises a head and a stack, wherein the stack comprises previously parsed tokens, and wherein the head comprises a current token;
• create, for each conditional token in the plurality of tokens, a token-follow set consisting of each possible language token that can immediately follow the conditional token;
• determine, for each conditional token, whether to fork a subparser for the conditional token in each of the token-follow sets, wherein the subparser parses a portion of the plurality of tokens including a token from the token-follow set of the conditional token;
• fork one or more subparsers based upon determining whether to fork a subparser for the conditional token in each of the token-follow sets, wherein each of the one or more subparsers have the same parser stack and comprises one or more previously parsed tokens up to and including the conditional token used to fork the one or more subparsers, wherein each of the one or more subparsers includes a presence condition based on the conditional token and a head that are unique to other subparsers, wherein each subparser has a head that is a token from the token-follow set, and wherein each subparser represents a distinct configuration associated with the presence condition;
• merge two or more subparsers into a merged parser when the two or more subparsers have the same head and stack, wherein a presence condition for the merged parser is a disjunction of the presence conditions of the two or more subparsers, and wherein a head of the merged parser is the head of the two or more subparsers;
• and
• generate a syntax tree from the plurality of tokens that preserves all possible configurations based upon the subparsers and the merged parser.

A method comprising:

• generating, by a lexer, a plurality of tokens from one or more source files, the one or more source files including source code in a first programming language, the source code comprising one or more static conditionals that include a conditional expression and branch code that is operative when the conditional expression is true, and wherein various configurations are possible based upon the conditionals;
• determining, by a configuration-preserving preprocessor module, that a first static conditional includes one or more nested static conditionals within the branch code associated with the first static conditional;
• and
• hoisting, by the configuration-preserving preprocessor module, each of the one or more nested static conditionals to a beginning of the branch code associated with the first static conditional, wherein each innermost branch code does not contain a static conditional, and wherein each possible configuration is preserved;
• and
• parsing, by a parser, a plurality of tokens, wherein the plurality of tokens comprise language tokens and conditional tokens, wherein various configurations are possible based upon the conditional tokens, wherein the parser comprises a head and a stack, wherein the stack comprises previously parsed tokens, and wherein the head comprises a current token;
• creating, by the parser, for each conditional token in the plurality of tokens, a token-follow set consisting of each possible language token that can immediately follow the conditional token;
• determining, by the parser, for each conditional token, whether to fork a subparser for the conditional token in each of the token-follow sets, wherein the subparser parses a portion of the plurality of tokens including a token from the token-follow set of the conditional token;
• forking, one or more subparsers based upon determining whether to fork a subparser for the conditional token in each of the token-follow sets, wherein each of the one or more subparsers have the same parser stack and comprises one or more previously parsed tokens up to and including the conditional token used to fork the one or more subparsers, wherein each of the one or more subparsers includes a presence condition based on the conditional token and a head that are unique to other subparsers, wherein each subparser has a head that is a token from the token-follow set, and wherein each subparser represents a distinct configuration associated with the presence condition;
• merging two or more subparsers into a merged parser when the two or more subparsers have the same head and stack, wherein a presence condition for the merged parser is a disjunction of the presence conditions of the two or more subparsers, and wherein a head of the merged parser is the head of the two or more subparsers;
• and
• generating a syntax tree from the plurality of tokens that preserves all possible configurations based upon the subparsers and the merged parser.