Unicode normalization
AssigneeMicrosoft Corporation
InventorShawn Steele
Life of this patent
Estimating the term…
Abstract
A system and process for achieving faster Unicode normalizations is described. Various approaches are highlighted that when used alone or in combination eliminate a number of time consuming routines of conventional normalization approaches. Some of the approaches include late normalization, avoiding second table lookups, compressed classes and use of flags, assigning classes to base characters, complicated characters, and flags.
Claims28
A process for normalizing Unicode comprising the following steps:
- examining each character;
- upon encountering a combinable character, determining whether a previous character is combinable with the character;
- normalizing the order of the character with the previous character, otherwise proceeding to the next character.
A process for normalizing Unicode comprising the following steps:
- examining each character;
- determining if the character is in a quick lookup table;
- skipping to the next character if the character is in the quick lookup table, otherwise, normalizing the character with a previous character.
A process for normalizing Unicode comprising the following steps:
- examining each character;
- determining if the character and a previous character are a special case;
- handling the special case;
- otherwise, normalizing said character and said previous character.
A process for normalizing Unicode comprising the following steps:
- examining each character;
- skipping the character if said character is a base character;
- otherwise, performing a full normalization of said character and a previous character.
A process for normalizing Unicode comprising the following steps:
- examining each character;
- skipping said character if said character is a low Unicode character;
- otherwise, looking up said character in a table and skipping said character if possible;
- otherwise, skipping said character if said character is one of special case characters;
- otherwise, skipping said character if said character is a base character;
- otherwise, performing a full normalization on said character.
A computer readable medium having computer-implemented instructions stored thereon, said instructions for normalizing Unicode, said instructions comprising the following steps:
- examining each character;
- upon encountering a combinable character, determining whether a previous character is combinable with the character;
- normalizing the order of the character with the previous character, otherwise proceeding to the next character.
A computer readable medium having computer-implemented instructions stored thereon, said instructions for normalizing Unicode, said instructions comprising the following steps:
- examining each character;
- determining if the character is in a quick lookup table;
- skipping to the next character if the character is in the quick lookup table, otherwise, normalizing the character with a previous character.
A computer readable medium having computer-implemented instructions stored thereon, said instructions for normalizing Unicode, said instructions comprising the following steps:
- examining each character;
- determining if the character and a previous character are a special case;
- handling the special case;
- otherwise, normalizing said character and said previous character.
A computer readable medium having computer-implemented instructions stored thereon, said instructions for normalizing Unicode, said instructions comprising the following steps:
- examining each character;
- skipping the character if said character is a base character;
- otherwise, performing a full normalization of said character and a previous character.
A computer readable medium having computer-implemented instructions stored thereon, said instructions for normalizing Unicode, said instructions comprising the following steps:
- examining each character;
- skipping said character if said character is a low Unicode character;
- otherwise, looking up said character in a table and skipping said character if possible;
- otherwise, skipping said character if said character is one of special case characters;
- otherwise, skipping said character if said character is a base character;
- otherwise, performing a full normalization on said character.
A computer system for normalizing Unicode, said system comprising:
- an input for receiving characters;
- a processor for normalizing said characters, said processor performing the steps of:
- examining each character;
- upon encountering a combinable character, determining whether a previous character is combinable with the character;
- normalizing the order of the character with the previous character, otherwise proceeding to the next character;
- an output for outputting a normalized form of said received characters.
A computer system for normalizing Unicode, said system comprising:
- an input for receiving characters;
- a processor for normalizing said characters, said processor performing the steps of:
- examining each character;
- determining if the character is in a quick lookup table;
- skipping to the next character if the character is in the quick lookup table, otherwise, normalizing the character with a previous character;
- an output for outputting a normalized form of said received characters.
A computer system for normalizing Unicode, said system comprising:
- an input for receiving characters;
- a processor for normalizing said characters, said processor performing the steps of:
- examining each character;
- determining if the character and a previous character are a special case;
- handling the special case;
- otherwise, normalizing said character and said previous character;
- an output for outputting a normalized form of said received characters.
A computer system for normalizing Unicode, said system comprising:
- an input for receiving characters;
- a processor for normalizing said characters, said processor performing the steps of:
- examining each character;
- skipping the character if said character is a base character;
- otherwise, performing a full normalization of said character and a previous character;
- an output for outputting a normalized form of said received characters.
A computer system for normalizing Unicode, said system comprising:
- an input for receiving characters;
- a processor for normalizing said characters, said processor performing the steps of:
- examining each character;
- skipping said character if said character is a low Unicode character;
- otherwise, looking up said character in a table and skipping said character if possible;
- otherwise, skipping said character if said character is one of special case characters;
- otherwise, skipping said character if said character is a base character;
- otherwise, performing a full normalization on said character;
- an output for outputting a normalized form of said received characters.
A computer system for normalizing Unicode comprising:
- means for examining each character;
- means for, upon encountering a combinable character, determining whether a previous character is combinable with the character;
- means for normalizing the order of the character with the previous character, otherwise proceeding to the next character.
A computer system for normalizing Unicode comprising the following steps:
- means for examining each character;
- means for determining if the character is in a quick lookup table;
- means for skipping to the next character if the character is in the quick lookup table, otherwise, normalizing the character with a previous character.
A computer system for normalizing Unicode comprising the following steps:
- means for examining each character;
- means for determining if the character and a previous character are a special case;
- means for handling the special case;
- otherwise, normalizing said character and said previous character.
A computer system for normalizing Unicode comprising the following steps:
- means for examining each character;
- means for skipping the character if said character is a base character;
- otherwise, performing a full normalization of said character and a previous character.
Dependencies are inferred from the claim text.