# US 9,436,999 B2 · Automatic image orientation and straightening through image analysis > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,436,999 B2 - Document ID: US9436999B2 - Status: Granted - Kind: B2 - Application number: 14/859,017 - Filed: 2015-09-18 - Published: 2016-01-14 - Granted: 2016-09-06 - Earliest claimed: 2012-09-11 - Assignees: Apple Inc. - Applicants: Apple Inc. - Inventors: Avi Cieplinski; Jan Erik Solem; Jerremy Holland; Garrett Johnson; Russell Reid; Russell Webb - Web page: https://patentgrep.com/patent/US9436999B2 ## CPC - G06T 11/23: using straight lines or curves - G06T 3/60: Rotation of whole images or parts thereof - G06T 7/30: Determination of transform parameters for the alignment of images, i.e. image registration - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06T 7/73: using feature-based methods - G06T 2207/20081: Training; Learning - G06T 2207/30201: Face - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Systems, methods, and computer readable media for adjusting the orientation of an image frame and a scene depicted in the image frame are described. In general, techniques are disclosed for analyzing an image with one or more feature detectors to identify features in the image. An alignment or position associated with one or more features identified in the image may be used to determine a proper orientation for the image frame. The image can then be rotated to the proper orientation. It may also be determined if a scene depicted in the image is properly aligned in the rotated image orientation. If not, alignment information associated with the identified features may be utilized to straighten the depicted scene. ## Claims ### Claim 1 A non-transitory program storage device, readable by a processor and comprising instructions stored thereon to cause the processor to: receive an image; determine an orientation for the received image based, at least in part, on one or more identified features in the received image; identify an alignment of the one or more identified features in the received image; determine an offset between a known typical alignment and the identified alignment for each of the one or more identified features at the determined orientation; rotate the received image based, at least in part, on the determined offset to generate a straightened image; and save the straightened image in a memory. ### Claim 2 (depends on 1) The non-transitory program storage device of claim 1, further comprising instructions to cause the processor to analyze the received image with one or more feature detectors in a predetermined order. ### Claim 3 (depends on 2) The non-transitory program storage device of claim 2, wherein the predetermined order comprises an order based on feature detector computational efficiency. ### Claim 4 (depends on 2) The non-transitory program storage device of claim 2, wherein the predetermined order comprises an order based on feature detector accuracy. ### Claim 5 (depends on 1) The non-transitory program storage device of claim 1, wherein the instructions to cause the processor to determine an orientation for the received image further comprise instructions to cause the processor to assign an orientation confidence value to the determined orientation. ### Claim 6 (depends on 5) The non-transitory program storage device of claim 5, wherein the orientation confidence value comprises a detected feature confidence value associated with each of the one or more identified features. ### Claim 7 (depends on 5) The non-transitory program storage device of claim 5, wherein the orientation confidence value comprises a predetermined accuracy value associated with one or more feature detectors. ### Claim 8 (depends on 1) The non-transitory program storage device of claim 1, further comprising instructions to cause the processor to rotate the received image to the determined orientation. ### Claim 9 (depends on 1) The non-transitory program storage device of claim 1, wherein the instructions to cause the processor to determine an orientation for the received image comprise instructions to cause the processor to determine a difference between a known typical position of the one or more identified features and a position of the one or more identified features in the received image. ### Claim 10 (depends on 1) The non-transitory program storage device of claim 1, wherein the instructions to cause the processor to determine an orientation for the received image comprise instructions to cause the processor to determine a difference between a known typical alignment of the one or more identified features and an alignment of the one or more identified features in the received image. ### Claim 11 (depends on 1) The non-transitory program storage device of claim 1, wherein the instructions to cause the processor to determine an orientation for the received image comprise instructions to cause the processor to utilize rotational information from an image capture device used to obtain the received image and an orientation determination for a previously analyzed image obtained by an image capture device. ### Claim 12 A device comprising: an image capture component; a memory; and a processor operatively coupled to the memory and the image capture component and configured to execute program code stored in the memory to: receive an image obtained with the image capture component; analyze the received image to identify a detected feature; determine an orientation for the received image based, at least in part, on the detected feature; determine an offset between a known typical alignment and an identified alignment of the detected feature at the determined orientation; rotate the image based, at least in part, on the determined offset to generate a straightened image; and save the straightened image in the memory. ### Claim 13 (depends on 12) The device of claim 12, wherein the detected feature comprises one or more human faces. ### Claim 14 (depends on 12) The device of claim 12, wherein the detected feature comprises intensity gradient information in the received image. ### Claim 15 (depends on 12) The device of claim 12, wherein the detected feature is identified based on a ranking of a plurality of available features. ### Claim 16 (depends on 12) The device of claim 12, wherein the program code to cause the processor to determine an orientation for the received image comprises program code to cause the processor to assign an orientation confidence value to the determined orientation. ### Claim 17 (depends on 16) The device of claim 16, further comprising program code to cause the processor to rotate the received image to the determined orientation. ### Claim 18 A method, comprising: receiving, by a processor, an image obtained with an image capture device; analyzing, by the processor, the received image with a feature detector; determining, by the processor, a first image orientation for the received image based, at least in part, on a detected alignment of one or more features in the received image identified by the feature detector; rotating the received image to the first image orientation; determining, by the processor, a second image orientation for the received image based, at least in part, on a difference between a known typical alignment and the first image orientation; rotating, by the processor, the received image based, at least in part, on the determined second image orientation to generate a straightened image; and saving the straightened image in a memory. ### Claim 19 (depends on 18) The method of claim 18, wherein determining a second image orientation for the received image comprises determining an in-plane rotational difference between a known typical alignment and the first image orientation. ### Claim 20 (depends on 18) The method of claim 18, wherein the feature detector comprises a feature detector configured to identify human faces.