Systems and methods for augmented reality video generation

#This disclosure is directed to systems and methods for augmented reality image and video generation and in particular for modification of chromakey (e.g., green screen) background replacement based on movement of a physical camera.

#Tools for end-user video content creation are becoming increasingly powerful. In one approach video sharing platforms may provide features built into their applications (as well as extensive developer kits) that allow for a user interface that provides an option to quickly modify a video (e.g., before it's shared). For example, user interface options may be provided to create filters, add animated sprites, and even perform chroma key (background replacement) effects.

#However, in such an approach visual effects tools merely mimic relatively low-end professional style tools, and do not take into account the ability of modern mobile devices to accurately determine and track changes in their positions and orientations in space with fine-grained resolution. In one approach, when chroma key is used, a green screen background may be replaced with computer-generated content using simple one-to-one pixel replacement, that does not take into account any motion of the camera, such as panning, zooming, or change of orientation.

#The present disclosure, in accordance with one or more various embodiments, is described in detail with reference to the following figures. The drawings are provided for purposes of illustration only and merely depict typical or example embodiments. These drawings are provided to facilitate an understanding of the concepts disclosed herein and should not be considered limiting of the breadth, scope, or applicability of these concepts. It should be noted that for clarity and case of illustration, these drawings are not necessarily made to scale.

#FIG. 1 shows an exemplary implementation of augmented reality image generation, in accordance with some embodiments of this disclosure;

#FIG. 2 shows another exemplary implementation of augmented reality image generation, in accordance with some embodiments of this disclosure;

#The term “physical camera” may refer to any suitable device capable of capturing the video content. For example, a physical camera may be a camera of a mobile phone with an integrated inertial measurement unit (IMU), and depth sensor. In another example, the physical camera may be any computing device capable of generating electric signals that represent detected light (e.g., using a complementary metal-oxide-semiconductor sensor and a lens).

#The term “Virtual camera” may refer to a viewpoint or perspective in a virtual environment from which a 3D scene is rendered.

#The term “physical background” may refer to a backdrop to which a physical camera is pointed to. In some examples, the physical background may be a physical screen painted a single color (e.g., green). In another example, the physical background may refer to all captured pixels by the physical camera other pixels of a detected image of a human face.