Decoder, encoder, bitstream generator, decoding method, and encoding method

A decoder comprising:

• memory;
• and
• circuitry coupled to the memory, wherein
• in operation, the circuitry:
• decodes, from one or more streams, (i) a fundamental image that is an image including a face and (ii) geometric information indicating geometric attributes of a subject and corresponding to each of frames of a captured video by a camera;
• inputs the fundamental image and the geometric information to the generative model to obtain an intermediate face video from the generative model, the intermediate face video being a video including the face;
• and
• generates a synthesized face video by embedding, into a background region in the intermediate face video, a corresponding region in the fundamental image.

An encoder comprising:

• memory;
• and
• circuitry coupled to the memory, wherein
• in operation, the circuitry:
• encodes, into one or more streams, (i) a fundamental image that is an image including a face, (ii) geometric information indicating geometric attributes of a subject and corresponding to each of frames of a captured video by a camera, and (iii) background information regarding a background image, (i) the fundamental image, (ii) the geometric information, and (iii) the background information being for generating a synthesized face video that is a video including a face and synthesized with the background image.

An encoder comprising:

• memory;
• and
• circuitry coupled to the memory, wherein
• in operation, the circuitry:
• encodes, into one or more streams, (i) a fundamental image that is an image including a face and (ii) geometric information indicating geometric attributes of a subject and corresponding to each of frames of a captured video by a camera, (i) the fundamental image and (ii) the geometric information being for generating a synthesized face video, and
• in generating the synthesized face video, the fundamental image and the geometric information are used to obtain an intermediate face video from a generative model by inputting the fundamental image and the geometric information to the generative model, and the fundamental image is further used to generate the synthesized face video by embedding, into a background region in the intermediate face video, a corresponding region in the fundamental image, the intermediate face video being a video including the face.

A bitstream generator comprising:

• memory;
• and
• circuitry coupled to the memory, wherein
• in operation, the circuitry:
• generates a bitstream including:
• (i) a fundamental image that is an image including a face;
• (ii) geometric information indicating geometric attributes of a subject and corresponding to each of frames of a captured video by a camera;
• and (iii) background information regarding a background image, (i) the fundamental image, (ii) the geometric information, and (iii) the background information being for generating a synthesized face video that is a video including a face and synthesized with the background image.