# US 6,297,825 B1 · Temporal smoothing of scene analysis data for image sequence generation > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 6,297,825 B1 - Document ID: US6297825B1 - Status: Granted - Kind: B1 - Application number: 09/055,843 - Filed: 1998-04-06 - Granted: 2001-10-02 - Earliest claimed: 1998-04-06 - Assignees: SynaPix, Inc. - Inventors: Paul B. Madden; Wesley K. Cobb; Jean-Pierre Schott; David Askey; Kenneth J. Eyring - Web page: https://patentgrep.com/patent/US6297825B1 ## CPC - G06T 5/70: Denoising; Smoothing - G06T 5/50: using two or more images, e.g. averaging or subtraction - G06T 2207/10016: Video; Image sequence - G06T 2207/20016: Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform - G06T 2207/20182: Noise reduction or smoothing in the temporal domain; Spatio-temporal filtering - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A technique for temporal smoothing of results from a scene analysis process which creates a sequence of visually pleasing and acceptable images generated in whole or part from such results. The technique applies temporal smoothing across time-related sets of scene analysis results. The spatial smoothing can be applied at various steps in the process: to images in the original sequence, to intermediate or final results of scene analysis in either a pixel-oriented or geometric domain, or to the images generated in whole or part from the scene analysis results. In a preferred embodiment, different levels of smoothing are applied to different parts of the intermediate or final results. The differentiation can be done using image masks (for pixel-oriented results) or geometry selection techniques (for geometric results). This allows a higher level of smoothing in certain areas where the generated visual images will benefit from such additional smoothing, while avoiding over-smoothing in other areas where the smoothing would obscure critical details within the generated images. ## Claims ### Claim 1 A method for processing a sequence of source images of a visual scene comprising the steps of: (a) analyzing the sequence of images to develop a data model of selected characteristics of the scene; (b) selecting at least two time-based series of data sets from the data model of selected characteristics provided by the scene analysis process; (c) comparing corresponding parametric values of a given time series with corresponding parametric values of another times series data sets, and (d) applying a smoothing filter and scaling adjustments to the parametric values to modify the data sets, to obtain visually pleasing results in a design visualization application. ### Claim 2 (depends on 1) A method as in claim 1 wherein the step of selecting a time-based series of data sets selects data from the original source image sequence. ### Claim 3 (depends on 1) A method as in claim 1 wherein the step of selecting a time-based series of data sets selects data from intermediate results of the scene analysis selected from the group consisting of camera parameters, captured parametric data, mark up data related to the images, or adjustments to parametric data. ### Claim 4 (depends on 1) A method as in claim 1 wherein the step of selecting a time-based series of data sets selects data from results of the scene analysis. ### Claim 5 (depends on 4) A method as in claim 4 wherein the results are a depth map. ### Claim 6 (depends on 4) A method as in claim 4 wherein the results are a surface mesh. ### Claim 7 (depends on 1) A method as in claim 1 wherein the step of selecting at least two time-based series of data sets additionally comprises calculating an optical flow field. ### Claim 8 (depends on 7) A method as in claim 7 additionally comprising obtaining a parallax fit. ### Claim 9 (depends on 7) A method as in claim 7 additionally comprising deriving a gamma coefficient. ### Claim 10 (depends on 1) A method as in claim 1 further comprising, prior to the step of applying a smoothing filter, deriving a gamma coefficient for each of a multiplicity of pixels, the gamma coefficients being inversely proportional to the pixel depth of the pixels. ### Claim 11 (depends on 10) A method as in claim 10 wherein the step of applying a smoothing filter includes applying the smoothing filter to the gamma coefficients. ### Claim 12 (depends on 11) A method as in claim 11 further comprising, after the step of applying a smoothing filter, converting each gamma coefficient to an estimated pixel depth. ### Claim 13 (depends on 12) A method as in claim 12 further comprising applying the smoothing filter to the estimated pixel depth. ### Claim 14 (depends on 10) A method as in claim 10 further comprising converting each gamma coefficient to an estimated pixel depth. ### Claim 15 (depends on 14) A method as in claim 14 wherein the step of applying a smoothing filter includes applying the smoothing filter to the estimated pixel depth. ### Claim 16 A method for processing a sequence of source images of a visual scene comprising the steps of: (a) analyzing the sequence of images to develop a data model of selected characteristics of the scene; (b) selecting at least two time-based series of data sets from the data model of selected characteristics provided by the scene analysis process; (c) comparing corresponding parametric values of a given time series with corresponding parametric values of another times series data sets; (d) deriving a gamma coefficient for each of a multiplicity of pixels, the gamma coefficients being inversely proportional to the pixel depth of the pixels; and (e) modifying the parametric values and the gamma coefficients, to obtain visually pleasing results in a design visualization application. ### Claim 17 (depends on 16) A method as in claim 16, wherein the step of modifying includes applying a smoothing filter to the gamma coefficients. ### Claim 18 (depends on 17) A method as in claim 17, further comprising converting each gamma coefficient to an estimated pixel depth. ### Claim 19 (depends on 18) A method as in claim 18, further comprising applying the smoothing filter to the estimated pixel depth.