# US 9,519,479 B2 · Techniques for increasing vector processing utilization and efficiency through vector lane predication prediction > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,519,479 B2 - Document ID: US9519479B2 - Status: Granted - Kind: B2 - Application number: 14/082,652 - Filed: 2013-11-18 - Published: 2015-05-21 - Granted: 2016-12-13 - Earliest claimed: 2013-11-18 - Assignees: Globalfoundries Inc. - Applicants: Globalfoundries Inc. - Inventors: Hung Q. Le; Jose E. Moreira; Pratap C. Pattnaik; Brian W. Thompto; Jessica H. Tseng - Web page: https://patentgrep.com/patent/US9519479B2 ## CPC - G06F 9/30036: {Instructions to perform operations on packed data, e.g. vector, tile or matrix operations} - G06F 9/3832: {Value prediction for operands; operand history buffers} - G06F 9/3887: {controlled by a single instruction for multiple data lanes [SIMD]} - G06F 9/3888: {controlled by a single instruction for multiple threads [SIMT] in parallel} - G06F 9/38885: {Divergence aspects} - G06F 9/30072: {to perform conditional operations, e.g. using predicates or guards} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Techniques for increasing vector processing utilization and efficiency through use of unmasked lanes of predicated vector instructions for executing non-conflicting instructions are provided. In one aspect, a method of vector lane predication for a processor is provided which includes the steps of: fetching predicated vector instructions from a memory; decoding the predicated vector instructions; determining if a mask value of the predicated vector instructions is available and, if the mask value of the predicated vector instructions is not available, predicting the mask value of the predicated vector instructions; and dispatching the predicated vector instructions to only masked vector lanes. ## Claims ### Claim 1 A method of vector lane predication for a processor, comprising the steps of: fetching predicated vector instructions from a memory; decoding the predicated vector instructions; determining if a mask value of the predicated vector instructions is available and, if the mask value of the predicated vector instructions is not available, predicting the mask value of the predicated vector instructions; dispatching the predicated vector instructions to only masked vector lanes; and dispatching non-conflicting instructions to only unmasked vector lanes at a same time as the predicated vector instructions are dispatched to the masked vector lanes, wherein the mask value of the predicated vector instructions is predicted, the method further comprising determining whether the mask value of the predicated vector instructions that is predicted is correct and, if the mask value of the predicated vector instructions that is predicted is incorrect, removing the predicated vector instructions for which the mask value that is predicted is incorrect, and resuming the method at the fetching step; decrementing a safeguard counter each time it is determined that the mask value of the predicated vector instructions that is predicted is incorrect, wherein the safeguard counter is set at a dynamic maximum tolerance level that includes a maximum bound of incorrect predictions that will be tolerated by taking into account a performance improvement of the processor device by the correct prediction of the mask value and also taking into account a performance penalty of the processor device by the incorrect prediction of the mask value; and reverting to a baseline process when the safeguard counter reaches zero which dispatches a same predicated vector instruction to all lanes until the mask value is known. ### Claim 2 (depends on 1) The method of claim 1, further comprising the steps of: issuing the predicated vector instructions to functional units of the processor; executing the predicated vector instructions with the functional units of the processor; and writing back data resulting from executing the predicated vector instructions for all vector lanes including the masked and unmasked vector lanes to a register file. ### Claim 3 (depends on 2) The method of claim 2, wherein the determining whether the mask value of the predicated vector instructions that is predicted is correct is prior to issuing the predicated vector instructions. ### Claim 4 (depends on 3) The method of claim 3, further comprising the step of: checking whether the mask value of the predicated vector instructions that is predicted is equal to an actual produced value. ### Claim 5 (depends on 1) The method of claim 1, further comprising the step of: after a preset period of time, resuming the method at the fetching step, and resetting the safeguard counter to permit re-execution of the method for a different phase of a workload that occurs by changes in a behavior of the workload during execution of the method. ### Claim 6 A method of vector lane predication for a processor, comprising the steps of: fetching predicated vector instructions from a memory; decoding the predicated vector instructions; determining if a mask value of the predicated vector instructions is available and, if the mask value of the predicated vector instructions is not available, predicting the mask value of the predicated vector instructions; dispatching the predicated vector instructions to only masked vector lanes; dispatching non-conflicting instructions to only unmasked vector lanes at a same time as the predicated vector instructions are dispatched to the masked vector lanes; issuing the predicated vector instructions to functional units of the processor; executing the predicated vector instructions with the functional units of the processor; writing back data resulting from executing the predicated vector instructions for all vector lanes including the masked and unmasked vector lanes to a register file, wherein the mask value of the predicated vector instructions is predicted, the method further comprising the step of determining whether the mask value of the predicated vector instructions that is predicted is correct and, if the mask value of the predicated vector instructions that is predicted is incorrect, removing the predicated vector instructions for which the mask value that is predicted is incorrect, and resuming the method at the fetching step; decrementing a safeguard counter each time it is determined that the mask value of the predicated vector instructions that is predicted is incorrect, wherein the safeguard counter is set at a dynamic maximum tolerance level that includes a maximum bound of incorrect predictions that will be tolerated by taking into account a performance improvement of the processor device by the correct prediction of the mask value and also taking into account a performance penalty of the processor device by the incorrect prediction of the mask value; and reverting to a baseline process when the safeguard counter reaches zero which dispatches a same predicated vector instruction to all lanes until the mask value is known. ### Claim 7 (depends on 6) The method of claim 6, further comprising the step of: after a preset period of time, resuming the method at the fetching step, and resetting the safeguard counter. ### Claim 8 An apparatus for vector lane predication, the apparatus comprising: a memory; and at least one processor device, coupled to the memory, operative to: fetch predicated vector instructions from the memory; decode the predicated vector instructions; determine if a mask value of the predicated vector instructions is available and, if the mask value of the predicated vector instructions is not available, predicting the mask value of the predicated vector instructions; dispatch the predicated vector instructions to only masked vector lanes; and dispatch non-conflicting instructions to only unmasked vector lanes at a same time as the predicated vector instructions are dispatched to the masked vector lanes, wherein the mask value of the predicated vector instructions is predicted, the at least one processor device is further operative to determine whether the mask value of the predicated vector instructions that is predicted is correct and, if the mask value of the predicated vector instructions that is predicted is incorrect, removing the predicated vector instructions for which the mask value that is predicted is incorrect, resume the fetch predicated vector instructions from the memory; decrement a safeguard counter each time it is determined that the mask value of the predicated vector instructions that is predicted is incorrect, wherein the safeguard counter is set at a dynamic maximum tolerance level that includes a maximum bound of incorrect predictions that will be tolerated by taking into account a performance improvement of the processor device by the correct prediction of the mask value and also taking into account a performance penalty of the processor device by the incorrect prediction of the mask value; and revert to a baseline process when the safeguard counter reaches zero which dispatches a same predicated vector instruction to all lanes until the mask value is known. ### Claim 9 (depends on 8) The apparatus of claim 8, wherein the at least one processor device is further operative to: issue the predicated vector instructions to functional units of the processor; execute the predicated vector instructions with the functional units of the processor; and write back data resulting from executing the predicated vector instructions for all vector lanes including the masked and unmasked vector lanes to a register file. ### Claim 10 (depends on 9) The apparatus of claim 9, wherein the at least one processor device is further operative to: determine whether the mask value of the predicated vector instructions that is predicted is correct prior to issuing the predicated vector instructions. ### Claim 11 (depends on 8) The apparatus of claim 8, wherein the at least one processor device is further operative to: after a preset period of time, resume the fetch the predicated vector instructions from the memory and reset the safeguard counter. ### Claim 12 A non-transitory article of manufacture for vector lane predication for a processor, comprising a machine-readable recordable medium containing one or more programs which when executed implement the steps of: fetching predicated vector instructions from a memory; decoding the predicated vector instructions; determining if a mask value of the instructions is available and, if the mask value of the predicated vector instructions is not available, predicting the mask value of the predicated vector instructions; dispatching the predicated vector instructions to only masked vector lanes; dispatching non-conflicting instructions to only unmasked vector lanes at a same time as the predicated vector instructions are dispatched to the masked vector lanes; decrementing a safeguard counter each time it is determined that the mask value of the predicated vector instructions that is predicted is incorrect, wherein the safeguard counter is set at a dynamic maximum tolerance level that includes a maximum bound of incorrect predictions that will be tolerated by taking into account a performance improvement of the processor device by the correct prediction of the mask value and also taking into account a performance penalty of the processor device by the incorrect prediction of the mask value; reverting to a baseline process when the safeguard counter reaches zero which dispatches a same predicated vector instruction to all lanes until the mask value is known; and resetting the safeguard counter after a preset period of time to permit re-execution of the method for a different phase of a workload.