# US 10,121,229 B2 · Self-portrait enhancement techniques > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,121,229 B2 - Document ID: US10121229B2 - Status: Granted - Kind: B2 - Application number: 14/984,805 - Filed: 2015-12-30 - Published: 2016-04-21 - Granted: 2018-11-06 - Earliest claimed: 2013-08-30 - Assignees: A9.com, Inc. - Applicants: A9.com, Inc. - Inventors: Douglas Ryan Gray; Colin Jon Taylor; Xiaofan Lin - Web page: https://patentgrep.com/patent/US10121229B2 ## CPC - H04N 5/272: Means for inserting a foreground image in a background image, i.e. inlay, outlay - G06F 16/5838: {using colour} - G06T 5/30: Erosion or dilatation, e.g. thinning - G06T 7/11: Region-based segmentation - G06T 7/194: involving foreground-background segmentation - G06T 11/60: Creating or editing images; Combining images with text - G06V 40/16: Human faces, e.g. facial parts, sketches or expressions - H04N 23/64: {Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image} - G06T 2207/10004: Still image; Photographic image - G06T 2207/20221: Image fusion; Image merging - G06T 2207/30201: Face - H04N 23/698: for achieving an enlarged field of view, e.g. panoramic image capture - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract Systems and approaches are provided for optimizing self-portraiture. The background of the self-portrait can be enhanced by image registration or stitching techniques of images captured using one or more conventional cameras. Multiple standard resolution images can be stitched together to generate a panoramic or a composite image of a higher resolution. Foreground elements, such as one or more representations of users, can also be enhanced in various ways. The representations of the users can be composited to exclude undesirable elements, such as image data of one of the users extending her arm to capture the self-portrait. An ideal pose of the users can automatically be selected and other image enhancements, such as histogram optimization, brightness and contrast optimization, color-cast correction, or reduction or removal of noise, can automatically be performed to minimize user effort in capturing self-portraits. ## Claims ### Claim 1 A computer-implemented method, comprising: obtaining a plurality of images; determining one or more background elements from each of the plurality of images; determining a plurality of foreground elements from each of the plurality of images; generating a background image comprising a first composite of at least a portion of the one or more background elements; determining a score of each of the plurality of foreground elements according to one or more visual characteristics; obtaining a selection of at least one instance of the plurality of foreground elements from at least a subset of the plurality of images based at least in part on the determined score associated with the at least one instance of the plurality of foreground elements; and generating a composite self-portrait image by compositing the selected at least one of the plurality of foreground elements and the background image, the composite self-portrait image comprising the selected at least one of the plurality of foreground elements having the highest determined score with respect to each other foreground element of the plurality of foreground elements. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein the plurality of foreground elements include at least a portion of representations of one or more users. ### Claim 3 (depends on 1) The computer-implemented method of claim 1, wherein the one or more visual characteristics include one or more facial features corresponding to the representations of the one or more users. ### Claim 4 (depends on 1) The computer-implemented method of claim 1, further comprising: obtaining one or more selections relating to display of the background image, wherein the self-portrait image is rendered based at least in part upon the one or more selections. ### Claim 5 (depends on 4) The computer-implemented method of claim 4, wherein the one or more selections relating to display of the background image include at least one of a boundary of the background image or an exposure of the background image. ### Claim 6 A computing device, comprising: one or more processors; one or more cameras; memory including instructions that, when executed by the one or more processors, cause the computing device to: obtain a plurality of images; determine one or more background elements from each of the plurality of images; determine a plurality of foreground elements from each of the plurality of images; generate a background image comprising a first composite of at least a portion of the one or more background elements; determine a score of each of the plurality of foreground elements according to one or more visual characteristics; obtaining a selection of at least one instance of the plurality of foreground elements from at least a subset of the plurality of images based at least in part on the determined score associated with the at least one instance of the plurality of foreground elements; and generate a composite self-portrait image by compositing the selected at least one of the plurality of foreground elements and the background image, the composite self-portrait image comprising the selected at least one of the plurality of foreground elements having the highest determined score with respect to each other foreground element of the plurality of foreground elements. ### Claim 7 (depends on 6) The computing device of claim 6, wherein the plurality of foreground elements include at least a portion of representations of one or more users. ### Claim 8 (depends on 6) The computing device of claim 6, wherein the one or more visual characteristics include one or more facial features corresponding to the representations of the one or more users. ### Claim 9 (depends on 6) The computing device of claim 6, wherein the instructions, when executed to generate the background image, further enable the computing device to: generate a preliminary background image by stitching together at least the portion of the one or more background elements; and fill in at least a portion of the preliminary background image using an inpainting process. ### Claim 10 (depends on 6) The computing device of claim 6, wherein the instructions, when executed by the one or more processors, further enable the computing device to: obtain one or more selections relating to display of the background image, wherein the self-portrait image is rendered based at least in part upon the one or more selections. ### Claim 11 (depends on 10) The computing device of claim 10, wherein the one or more selections relating to display of the background image include at least one of a boundary of the background image or an exposure of the background image. ### Claim 12 A non-transitory computer-readable storage medium storing instructions which, when executed by a processor, cause the processor to: obtain a plurality of images; determine one or more background elements from each of the plurality of images; determine a plurality of foreground elements from each of the plurality of images; generate a background image comprising a first composite of at least a portion of the one or more background elements; determine a score of each of the plurality of foreground elements according to one or more visual characteristics; obtain a selection of at least one instance of the plurality of foreground elements from at least a subset of the plurality of images based at least in part on the determined score associated with the at least one instance of the plurality of foreground elements; and generate a composite self-portrait image by compositing the selected at least one of the plurality of foreground elements and the background image, the composite self-portrait image comprising the selected at least one of the plurality of foreground elements having the highest determined score with respect to each other foreground element of the plurality of foreground elements. ### Claim 13 (depends on 12) The non-transitory computer-readable storage medium of claim 12, wherein the plurality of foreground elements include at least a portion of representations of one or more users. ### Claim 14 (depends on 12) The non-transitory computer-readable storage medium of claim 12, wherein the one or more visual characteristics include one or more facial features corresponding to the representations of the one or more users. ### Claim 15 (depends on 12) The non-transitory computer-readable storage medium of claim 12, wherein the instructions, when executed to generate the background image, further cause the processor to: generate a preliminary background image by stitching together at least the portion of the one or more background elements; and fill in at least a portion of the preliminary background image using an inpainting process. ### Claim 16 (depends on 12) The non-transitory computer-readable storage medium of claim 12, wherein the instructions, when executed by the processor, further cause the processor to: obtain one or more selections relating to display of the background image, wherein the one or more selections relating to display of the background image include at least one of a boundary of the background image or an exposure of the background image; and wherein the self-portrait image is rendered based at least in part upon the one or more selections.