# US 12,223,657 B2 · Image segmentation system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,223,657 B2 - Document ID: US12223657B2 - Status: Granted - Kind: B2 - Application number: 18/136,212 - Filed: 2023-04-18 - Published: 2023-08-10 - Granted: 2025-02-11 - Earliest claimed: 2019-03-26 - Assignees: Snap Inc. - Applicants: Snap Inc. - Inventors: Menglei Chai; David LeMieux; Shubham Vij; Ian Wehrman - Web page: https://patentgrep.com/patent/US12223657B2 ## CPC - G06T 7/11: Region-based segmentation - G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour - G06F 3/0488: using a touch-screen or digitiser, e.g. input of commands through traced gestures - G06T 7/12: Edge-based segmentation - G06T 7/13: Edge detection - G06T 7/174: involving the use of two or more images - G06T 7/194: involving foreground-background segmentation - G06V 10/56: relating to colour - G06T 2207/20104: Interactive definition of region of interest [ROI] - G06T 2207/20132: Image cropping - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract An image segmentation system to perform operations that include causing display of an image within a graphical user interface of a client device, receive a set of user inputs that identify portions of a background and foreground of the image, identify a boundary of an object depicted within the image based on the set of user inputs, crop the object from the image based on the boundary, and generate a media item based on the cropped object, wherein properties of the media object, such as a size and a shape, are based on the boundary of the object. ## Claims ### Claim 1 A method comprising: causing display of an image within a graphical user interface at a client device, the image comprising a display of an object, a background, and a foreground; receiving an input that identifies a portion of the image that corresponds with the foreground of the image; causing display of a visual representation of the input; identifying a boundary of the display of the object depicted within the foreground of the image based on a position of the visual representation of the input; cropping the object from the image based on the boundary of the object; and generating a media object based on the object, the media object comprising a shape based on the boundary of the object. ### Claim 2 (depends on 1) The method of claim 1, further comprising: saving the media object at a memory location associated with a user of the client device. ### Claim 3 (depends on 1) The method of claim 1, wherein the image is a first image, and the method further comprises: causing display of a second image at the client device; receiving a tactile input that identifies a position within the second image; and presenting the media object at the position within the second image based on the tactile input that identifies the position within the second image. ### Claim 4 (depends on 3) The method of claim 3, further comprising: generating a message that includes the second image that includes the media object at the position within the second image. ### Claim 5 (depends on 1) The method of claim 1, wherein the receiving the input that identifies the foreground of the image includes: receiving the input comprises an identification of a set of pixels from within the foreground portion of the image; generating a color model of the set of pixels; and identifying the foreground of the image based on the color model. ### Claim 6 (depends on 1) The method of claim 1, wherein the input that identifies the foreground of the image comprises a tactile input that draws a line over the foreground of the image. ### Claim 7 (depends on 1) The method of claim 1, wherein the input that identifies the foreground of the image comprises a first input that identifies a first set of pixels within the foreground of the image and a second input that identifies a second set of pixels within a background of the image, and the identifying the boundary of the display of the object includes: generating a first color model that represents the foreground portion of the image based on the first set of pixels; generating a second color model that represents the background portion of the image based on the second set of pixels; and identifying the boundary of the object depicted in the images based on the first color model and the second color model. ### Claim 8 A system comprising: a memory; and at least one hardware processor coupled to the memory and comprising instructions that causes the system to perform operations comprising: causing display of an image within a graphical user interface at a client device, the image comprising a display of an object, a background, and a foreground; receiving an input that identifies a portion of the image that corresponds with the foreground of the image; causing display of a visual representation of the input; identifying a boundary of the display of the object depicted within the foreground of the image based on a position of the visual representation of the input; cropping the object from the image based on the boundary of the object; and generating a media object based on the object, the media object comprising a shape based on the boundary of the object. ### Claim 9 (depends on 8) The system of claim 8, further comprising: saving the media object at a memory location associated with a user of the client device. ### Claim 10 (depends on 8) The system of claim 8, wherein the image is a first image, and the method further comprises: causing display of a second image at the client device; receiving a tactile input that identifies a position within the second image; and presenting the media object at the position within the second image based on the tactile input that identifies the position within the second image. ### Claim 11 (depends on 10) The system of claim 10, further comprising: generating a message that includes the second image that includes the media object at the position within the second image. ### Claim 12 (depends on 8) The system of claim 8, wherein the receiving the input that identifies the foreground of the image includes: receiving the input comprises an identification of a set of pixels from within the foreground portion of the image; generating a color model of the set of pixels; and identifying the foreground of the image based on the color model. ### Claim 13 (depends on 8) The system of claim 8, wherein the input that identifies the foreground of the image comprises a tactile input that draws a line over the foreground of the image. ### Claim 14 (depends on 8) The system of claim 8, wherein the input that identifies the foreground of the image comprises a first input that identifies a first set of pixels within the foreground of the image and a second input that identifies a second set of pixels within a background of the image, and the identifying the boundary of the display of the object includes: generating a first color model that represents the foreground portion of the image based on the first set of pixels; generating a second color model that represents the background portion of the image based on the second set of pixels; and identifying the boundary of the object depicted in the images based on the first color model and the second color model. ### Claim 15 A non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors of a machine, cause the machine to perform operations comprising: causing display of an image within a graphical user interface at a client device, the image comprising a display of an object, a background, and a foreground; receiving an input that identifies a portion of the image that corresponds with the foreground of the image; causing display of a visual representation of the input; identifying a boundary of the display of the object depicted within the foreground of the image based on a position of the visual representation of the input; cropping the object from the image based on the boundary of the object; and generating a media object based on the object, the media object comprising a shape based on the boundary of the object. ### Claim 16 (depends on 15) The non-transitory machine-readable storage medium of claim 15, further comprising: saving the media object at a memory location associated with a user of the client device. ### Claim 17 (depends on 15) The non-transitory machine-readable storage medium of claim 15, wherein the image is a first image, and the method further comprises: causing display of a second image at the client device; receiving a tactile input that identifies a position within the second image; and presenting the media object at the position within the second image based on the tactile input that identifies the position within the second image. ### Claim 18 (depends on 17) The non-transitory machine-readable storage medium of claim 17, further comprising: generating a message that includes the second image that includes the media object at the position within the second image. ### Claim 19 (depends on 15) The non-transitory machine-readable storage medium of claim 15, wherein the receiving the input that identifies the foreground of the image includes: receiving the input comprises an identification of a set of pixels from within the foreground portion of the image; generating a color model of the set of pixels; and identifying the foreground of the image based on the color model. ### Claim 20 (depends on 15) The non-transitory machine-readable storage medium of claim 15, wherein the input that identifies the foreground of the image comprises a tactile input that draws a line over the foreground of the image.