# US 12,008,728 B2 · Apparatuses and methods for processing single instruction for image transformation from non-integral locations > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,008,728 B2 - Document ID: US12008728B2 - Status: Granted - Kind: B2 - Application number: 17/823,711 - Filed: 2022-08-31 - Published: 2024-02-29 - Granted: 2024-06-11 - Earliest claimed: 2022-08-31 - Assignees: QUALCOMM Incorporated - Applicants: QUALCOMM Incorporated - Inventors: Venkata Prema Sai Sravan Patchala; Mithil Ramteke; Sridhar Kandimalla; Himanshu Pradeep Aswani - Web page: https://patentgrep.com/patent/US12008728B2 ## CPC - G06T 3/40: Scaling of whole images or parts thereof, e.g. expanding or contracting - G06T 3/4007: based on interpolation, e.g. bilinear interpolation (image demosaicing G06T3/4015; edge-driven or edge-based scaling G06T3/403) - G06F 9/3001: {Arithmetic instructions} - G06F 9/30036: {Instructions to perform operations on packed data, e.g. vector, tile or matrix operations} - G06F 9/3555: {using scaling, e.g. multiplication of index} - G06F 9/3887: {controlled by a single instruction for multiple data lanes [SIMD]} - G06T 1/60: Memory management - G06T 5/77: Retouching; Inpainting; Scratch removal - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A processor pipeline circuit in a processor for non-integral transformation of an image utilizing a single instruction is disclosed. The processor pipeline circuit comprises a data fetch circuit configured to receive a memory address of the input image and fetch a plurality of pixels of the input image. The processor pipeline circuit further comprises a weights access circuit configured to receive an element of the array of offsets and the interpolation type parameter. The weights access circuit is configured to determine weights to be applied to the plurality of pixels of the input image. The processor pipeline circuit further comprises a multiply and add circuit configured to calculate the output pixel of the transformed image by multiplying the plurality of pixels of the input image by the weights and summing each resulting product. ## Claims ### Claim 1 An apparatus for non-integral transformation of an image, comprising: a processor pipeline circuit, the processor pipeline circuit configured to receive a single instruction containing a memory address of an input image, a second memory address of an output image, an array of offsets associated with the output image, and an interpolation type parameter, the processor pipeline circuit configured to write an output pixel of a transformed image, the processor pipeline circuit comprising: a data fetch circuit configured to receive the memory address of the input image, the data fetch circuit further configured to fetch a plurality of pixels of the input image; a weights access circuit configured to receive an element of the array of offsets and the interpolation type parameter, the weights access circuit configured to determine weights to be applied to the plurality of pixels of the input image; a multiply and add circuit configured to calculate the output pixel of the transformed image by multiplying the plurality of pixels of the input image by the weights and summing each resulting product; and an output data circuit configured to receive the second memory address of the output image and the output pixel from the multiply and add circuit, the output data circuit configured to write the output pixel of the transformed image to memory. ### Claim 2 (depends on 1) The apparatus of claim 1, wherein the array of offsets include x,y coordinates where at least one axis has a floating point value. ### Claim 3 (depends on 1) The apparatus of claim 1, wherein the weights access circuit computes the weights based on the element of the array of offsets and the interpolation type parameter. ### Claim 4 (depends on 1) The apparatus of claim 1, wherein the weights access circuit determines the weights by retrieving the weights from memory based on the element of the array of offsets. ### Claim 5 (depends on 1) The apparatus of claim 1, wherein the output pixel comprises three channels, the three channels including a red channel, a green channel, and a blue channel. ### Claim 6 (depends on 1) The apparatus of claim 1, further comprising a multiplexer circuit coupled to the data fetch circuit with n lines, the multiplexer circuit configured to receive the interpolation type parameter and to select an active subset of n lines carrying the plurality of pixels of the input image to the multiply and add circuit, each active line representing one pixel of the plurality of pixels of the input image. ### Claim 7 (depends on 1) The apparatus of claim 1, wherein the interpolation type parameter is a bilinear 1×2 interpolation, bilinear 2×1 interpolation, bilinear 2×2 interpolation, bilinear 3×3 interpolation, or a multi-linear interpolation. ### Claim 8 (depends on 6) The apparatus of claim 6, wherein the interpolation type parameter is bilinear 1×2 interpolation, and wherein the active subset of n lines is 2 lines. ### Claim 9 (depends on 6) The apparatus of claim 6, wherein the interpolation type parameter is bilinear 2×2 interpolation, and wherein the active subset of n lines is 4 lines. ### Claim 10 (depends on 6) The apparatus of claim 6, wherein the interpolation type parameter is bilinear 3×3 interpolation, and wherein the active subset of n lines is 9 lines. ### Claim 11 A method for non-integral transformation of an image, comprising: receiving a single instruction containing a memory address of an input image, a second memory address of an output image, an array of offsets associated with the output image, and an interpolation type parameter; fetching a plurality of pixels of the input image from memory based on the memory address of the input image; determining weights to be applied to the plurality of pixels of the input image based on an element of the array of offsets and the interpolation type parameter; calculating an output pixel of a transformed image by multiplying the plurality of pixels of the input image by the weights and summing each resulting product; and writing the output pixel of the transformed image to the second memory address. ### Claim 12 (depends on 11) The method of claim 11, wherein the array of offsets include x,y coordinates where at least one axis has a floating point value. ### Claim 13 (depends on 11) The method of claim 11, wherein the interpolation type parameter is a bilinear 1×2 interpolation, bilinear 2×1 interpolation, bilinear 2×2 interpolation, bilinear 3×3 interpolation, or multi-linear interpolation. ### Claim 14 (depends on 11) The method of claim 11, wherein the interpolation type parameter is bilinear 1×2 interpolation. ### Claim 15 (depends on 11) The method of claim 11, wherein the interpolation type parameter is bilinear 2×2 interpolation. ### Claim 16 A non-transitory computer-readable storage medium comprising instructions executable by a processor, which when executed by the processor causes the processor to perform an image transformation utilizing a single instruction multiple data (SIMD) operation, the non-transitory computer-readable storage medium, comprising: a SIMD instruction which specifies: a memory address of an input image; a second memory address of an output image; an array of offsets associated with the output image; an interpolation type parameter; and code for fetching a plurality of pixels of the input image based on the memory address of the input image; code for determining weights to be applied to the plurality of pixels of the input image based on an element of the array of offsets and the interpolation type parameter; code for calculating an output pixel of a transformed image by multiplying the plurality of pixels of the input image by the weights and summing each resulting product; and code for writing the output pixel of the transformed image to the second memory address. ### Claim 17 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein the interpolation type parameter is a bilinear 1×2 interpolation, bilinear 2×1 interpolation, bilinear 2×2 interpolation, bilinear 3×3 interpolation, or multi-linear interpolation. ### Claim 18 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein the interpolation type parameter is bilinear 1×2 interpolation. ### Claim 19 (depends on 16) The non-transitory computer-readable storage medium of claim 16, wherein the interpolation type parameter is bilinear 2×2 interpolation.