# US 2020/0394828 A1 · Image object pose synchronization

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2020/0394828 A1
- Document ID: US20200394828A1
- Status: Published application
- Kind: A1
- Application number: 16/438,079
- Filed: 2019-06-11
- Published: 2020-12-17
- Earliest claimed: 2019-06-11
- Assignees: Adobe Inc.
- Applicants: Adobe Inc.
- Inventors: Sankalp Shukla; Sourabh Gupta; Angad Kumar Gupta
- Web page: https://patentgrep.com/patent/US20200394828A1

## CPC

- G06F 3/0481: based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F 3/0482: Interaction with lists of selectable items, e.g. menus
- G06F 3/04845: for image manipulation, e.g. dragging, rotation, expansion or change of colour
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 11/60: Creating or editing images; Combining images with text
- G06T 17/20: Finite element generation, e.g. wire-frame surface description, {tesselation}
- G06V 40/10: Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V 40/103: {Static body considered as a whole, e.g. static pedestrian or occupant recognition}
- G06T 2207/30196: Human being; Person
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING

## Abstract

Techniques for automatically synchronizing poses of objects in an image or between multiple images. An automatic pose synchronization functionality is provided by an image editor. The image editor identifies or enables a user to select objects (e.g., people) whose poses are to be synchronized and the image editor then performs processing to automatically synchronize the poses of the identified objects. For two objects whose poses are to be synchronized, a reference object is identified as one whose associated pose is to be used as a reference pose. A target object is identified as one whose associated pose is to be modified to match the reference pose of the reference object. An output image is generated by the image editor in which the position of a part of the target object is modified such that the pose associated with the target object matches the reference pose of the reference object.

## Claims

### Claim 1

A computer-implemented method in which one or more computing systems perform operations comprising:
processing one or more images to determine a first pose of a first part of a first object in the one or more images;
and a second pose of a second part of a second object in the one or more images, wherein the second pose is different from the first pose;
changing, based upon the first pose and the second pose, a position of the second part of the second object from the second pose to a modified pose that is in synchronization with the first pose of the first part of the first object; and
modifying the image that includes the second object to generate an output image, wherein, in the output image, the second part of the second object is in the modified pose instead of in the second pose.

### Claim 2 (depends on 1)

The computer-implemented method of claim 1, wherein:
determining the first pose of the first part comprises:
identifying, in the one or more images, a first plurality of key points associated with the first object, the first plurality of key points defining a first plurality of parts of the first object including the first part;
determining the first pose of the first part by determining at least a first angle between the first part and a third part of the first plurality of parts of the first object;
determining the second pose of the second part comprises:
identifying, in the one or more images, a second plurality of key points associated with the second object, the second plurality of key points defining a second plurality of parts of the second object including the second part; and
determining the second pose of the second part by determining at least a second angle between the second part and a fourth part of the second plurality of parts of the second object; and
changing the position of the second part comprises changing the position of the second part such that an angle between the second part and the third part matches the first angle.

### Claim 3 (depends on 1)

The computer-implemented method of claim 1 wherein:
the first object is a first human;
the second object is a second human;
the first part is a body part of the first human; and
the second part is a body part of the second human.

### Claim 4 (depends on 3)

The computer-implemented method of claim 3, wherein the first part is an arm, a leg, a head, a neck, or a shoulder of the first human.

### Claim 5 (depends on 1)

The computer-implemented method of claim 1 wherein changing the position of the second part comprises:
determining a first angle associated with the first part;
determining a second angle associated with the second part;
determining that the second angle varies from the first angle by at least a threshold angle value; and
changing the position of the second part such that an angle associated with the second part matches the first angle.

### Claim 6 (depends on 5)

The computer-implemented method of claim 5 wherein changing the position of the second part further comprises:
determining a third angle associated with subparts of the first part;
determining a fourth angle associated with subparts of the second part; and
changing the position of the second part such that an angle associated with the subparts of the second part matches the third angle.

### Claim 7 (depends on 1)

The computer-implemented method of claim 1, wherein changing the position of the second part comprises:
generating a mesh model of the second part; and
changing the position of the second part to the modified pose based on the mesh model of the second part.

### Claim 8 (depends on 1)

The computer-implemented method of claim 1, wherein generating the output image comprising the second object comprises generating image content for an area previously occluded by the second part of the second object.

### Claim 9 (depends on 1)

The computer-implemented method of claim 1, further comprising:
receiving a user input requesting the second pose of the second part of the second object be synchronized with the first pose of the first part of the first object, wherein determining the first pose, determining the second pose, changing the position of the second part, and generating the output image are performed responsive to receiving the user input.

### Claim 10 (depends on 1)

The computer-implemented method of claim 1, wherein the first object and the second object are in a same image of the one or more images.

### Claim 11 (depends on 1)

The computer-implemented method of claim 1, wherein the first object and the second object are in two separate images of the one or more images.

### Claim 12 (depends on 1)

The computer-implemented method of claim 1, wherein the output image comprises the first object comprising the first part in the first pose.

### Claim 13 (depends on 1)

The computer-implemented method of claim 1, further comprising, prior to determining the first pose and determining the second pose:
providing a user interface configured to edit images from the one or more images, the user interface comprising a user-selectable option for requesting synchronization of poses associated with objects; and
receiving an indication of selection of the user-selectable option for synchronizing poses associated with the first object and the second object.

### Claim 14

A system comprising:
one or more processors; and
a non-transitory computer-readable medium communicatively coupled to the one or more processors, the non-transitory computer-readable medium storing program code executable by the one or more processors, the program code comprising instructions that when executed by the one or more processors cause the one or more processors to perform operations comprising:
processing determining, from an image to determine a first pose of a first part of a first object in the image
and a second pose of a second part of a second object in the image;
changing, based upon the first pose and the second pose, a position of the second part of the second object from the second pose to a modified pose that is in synchronization with the first pose of the first part of the first object; and
modifying the image to generate an output image comprising the second object, wherein, in the output image, the second part of the second object is in the modified pose instead of in the second pose.

### Claim 15 (depends on 14)

The system of claim 14, wherein
determining the first pose of the first part comprises:
identifying, in the image, a first plurality of key points associated with the first object, the first plurality of key points defining a first plurality of parts of the first object including the first part; and
determining the first pose of the first part by determining at least a first angle between the first part and a third part of the first plurality of parts of the first object;
determining the second pose of the second part comprises:
identifying, in the image, a second plurality of key points associated with the second object, the second plurality of key points defining a second plurality of parts of the second object including the second part; and
determining the second pose of the second part by determining at least a second angle between the second part and a fourth part of the second plurality of parts of the second object; and
changing the position of the second part comprises changing the position of the second part such that an angle between the second part and the third part matches the first angle.

### Claim 16 (depends on 14)

The system of claim 14, wherein changing the position of the second part comprises:
determining a first angle associated with the first part;
determining a second angle associated with the second part;
determining that the second angle varies from the first angle by at least a threshold angle value; and
changing the position of the second part such that an angle associated with the second part matches the first angle.

### Claim 17 (depends on 16)

The system of claim 16, wherein changing the position of the second part further comprises:
determining a third angle associated with subparts of the first part;
determining a fourth angle associated with subparts of the second part; and
changing the position of the second part such that an angle associated with the subparts of the second part matches the third angle.

### Claim 18

A non-transitory computer-readable medium having program code that is stored thereon, the program code executable by one or more processing devices for performing operations comprising:
processing an image to determine a first pose of a first part of a first object in the image;
and a second pose of a second part of a second object in the image;
changing, based upon the first pose and the second pose, a position of the second part of the second object from the second pose to a modified pose that is in synchronization with the first pose of the first part of the first object; and
modifying the image to generate an output image comprising the second object, wherein, in the output image, the second part of the second object is in the modified pose instead of in the second pose.

### Claim 19 (depends on 18)

The non-transitory computer-readable medium of claim 18,
wherein determining the first pose of the first part comprises:
identifying, in the image, a first plurality of key points associated with the first object, the first plurality of key points defining a first plurality of parts of the first object including the first part; and
determining the first pose of the first part by determining at least a first angle between the first part and a third part of the first plurality of parts of the first object;
wherein determining the second pose of the second part comprises:
identifying, in the image, a second plurality of key points associated with the second object, the second plurality of key points defining a second plurality of parts of the second object including the second part; and
determining the second pose of the second part by determining at least a second angle between the second part and a fourth part of the second plurality of parts of the second object; and
wherein changing the position of the second part comprises changing the position of the second part such that an angle between the second part and the third part matches the first angle.

### Claim 20 (depends on 18)

The non-transitory computer-readable medium of claim 18, wherein
changing the position of the second part comprises:
determining a first angle associated with the first part;
determining a second angle associated with the second part;
determining that the second angle varies from the first angle by at least a threshold angle value; and
changing the position of the second part such that an angle associated with the second part matches the first angle.
