System and method for real-time multi-planar image processing with depth estimation and image synthesis

Abstract

Real-time image processing uses multi-planar images in a multiple camera system. Images are downscaled and gray scaled to reduce computational load. A target-centered depth map is estimated by sweeping multiple layers of depth across the downscaled and gray scaled input images to form a plane sweep volume (PSV) and corresponding layers of the PSV are compared to generate a similarity volume, by using Laplacian pyramids, including decomposing the downscaled and gray scaled input images into Laplacian pyramids to extract fine and large edges. Structural features are compared using the Structural Similarity Index, SSIM, at multiple depth layers. Depth values are selected for each pixel based on the similarity volume. The downscaled and gray scaled source images are warped to a target viewpoint based on the estimated depth map and the warped images are blended to generate a synthesized target image.

Dependencies are inferred from the claim text.