Modifying video regions using mobile device input

A method, comprising:

• receiving input generated with a mobile device for positioning an edge segmentation template, the input being generated by:
• generating one or more face regions for one or more regions of the input video using a camera coupled to the mobile device, wherein:
• the face regions include a foreground face region and one or more background face regions;
• the foreground face region is assigned one or more weights associated with being designated as a portion of the foreground region;
• and
• the background face regions are assigned one or more weights associated with being designated as a portion of the background regions;
• and
• positioning the edge segmentation template based on the face regions;
• producing an initial representation for segmenting input video into a plurality of portions, the initial representation being based on the positioned edge segmentation template, the initial representation including weights for one or more regions of the input video to be designated as foreground regions or background regions;
• and
• based on the initial representation, segmenting the input video by designating one or more of the portions of the input video as foreground regions or background regions.

One or more computer-readable storage media storing computer-readable instructions that when executed by a computer, cause the computer to perform a method, the method comprising:

• receiving input generated with the mobile device for positioning an edge segmentation template, the template positioning input being generated by:
• displaying a template superimposed over video displayed on the display, and
• receiving user input for positioning the template relative to the displayed video, the user input including at least one or more of the following:
• moving the video camera, detecting position and/or movement of the mobile device, or receiving touch screen display input using the display;
• generating a positioned edge segmentation template based at least in part on the user input;
• producing an initial representation for segmenting input video into a plurality of portions, the initial representation being based on the positioned edge segmentation template, the initial representation including weights for one or more regions of the input video to be designated as foreground regions or background regions at least one of the weights being based on a term selected from the group of terms consisting of color likelihood, spatial prior likelihood, temporal prior likelihood, and motion likelihood;
• and
• based on the initial representation, segmenting the input video by designating one or more of the portions of the input video as foreground regions or background regions.

A method of designating background regions in a first image selected from a sequence of images, the method comprising:

• generating one or more energy terms for the first image based on an edge segmentation template positioned using a mobile device touch screen display, at least one of the energy terms being selected from the group of terms consisting of color likelihood, spatial prior likelihood, temporal prior likelihood, and motion likelihood;
• based on the energy terms, designating one or more regions of the first image as background regions;
• and
• replacing one or more of the designated background regions of the image with corresponding regions from a different image or video than the first image to produce a modified sequence of images.

A method of designating background regions in a first image selected from a sequence of images, the method comprising:

• generating one or more energy terms for the first image based on an edge segmentation template positioned using a mobile device, the one or more of the energy terms being based on seed regions for a second image in the sequence of images, the seed regions designating one or more pixels of the second image as either a foreground region or a background region based on user input received with the mobile device;
• based on the energy terms, designating one or more regions of the first image as background regions;
• and
• replacing one or more of the designated background regions of the image with corresponding regions from a different image or video than the first image to produce a modified sequence of images, the replacing the background regions comprising applying a filter to the designated background regions of the image to produce the modified sequence of images, the modified sequence of images comprising one or more regions of the image designated as foreground portions.

A mobile device, comprising:

• a video camera operable to produce input video;
• a touch screen display for receiving touch screen input and displaying video;
• at least one processor coupled to the video camera and the touch screen display;
• a communication interface coupled to the at least one processor;
• and
• one or more computer-readable storage media storing computer-readable instructions executable by the at least one processor for transmitting modified video based on the input video, the instructions comprising:
• instructions for receiving touch screen input from the touch screen display and designating one or more regions of the input video based on the touch screen input,
• instructions for designating one or more regions of a portion of the input video using a representation of a conditional random field (CRF),
• instructions for replacing one or more of the designated regions in the input video with corresponding regions from a second image or video to produce the modified video, wherein the replacing is based at least in part on the designated regions, and
• instructions for transmitting the modified video using the communication interface.

A mobile device, comprising:

• a video camera operable to produce input video;
• at least one processor coupled to the video camera;
• a communication interface coupled to the at least one processor;
• and
• one or more computer-readable storage media storing computer-readable instructions executable by the at least one processor for transmitting modified video based on the input video, the instructions comprising:
• instructions for generating one or more energy terms for a first image of the input video based on an edge segmentation template positioned using the mobile device, at least one of the energy terms being selected from the group of terms consisting of color likelihood, spatial prior likelihood, temporal prior likelihood, and motion likelihood;
• instructions for, based on the energy terms, designating one or more regions of the first image as background regions;
• and
• instructions for replacing one or more of the designated background regions of the image with corresponding regions from a different image or video than the first image to produce a modified sequence of images.

One or more computer-readable storage media storing computer-readable instructions that when executed by a computer, cause the computer to perform a method of designating background regions in a first image selected from a sequence of images, the method comprising:

• generating one or more energy terms for the first image based on an edge segmentation template positioned using a mobile device touch screen display, at least one of the energy terms being selected from the group of terms consisting of color likelihood, spatial prior likelihood, temporal prior likelihood, and motion likelihood;
• based on the energy terms, designating one or more regions of the first image as background regions;
• and
• replacing one or more of the designated background regions of the image with corresponding regions from a different image or video than the first image to produce a modified sequence of images.