# US 11,908,070 B2 · Dynamic three-dimensional imaging method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,908,070 B2 - Document ID: US11908070B2 - Status: Granted - Kind: B2 - Application number: 17/427,877 - Filed: 2020-02-03 - Published: 2022-04-21 - Granted: 2024-02-20 - Earliest claimed: 2020-02-03 - Applicants: Olivier Querbes; Véronique Querbes-Duret - Inventors: Olivier Querbes; Véronique Querbes-Duret - Web page: https://patentgrep.com/patent/US11908070B2 ## CPC - G06T 17/00: Three-dimensional [3D] modelling for computer graphics - G06T 7/593: from stereo images - G06T 7/90: Determination of colour characteristics - G06T 2207/10024: Color image - G06T 2207/10028: Range image; Depth image; 3D point clouds - G06T 2207/20081: Training; Learning - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Disclosed is a dynamic three-dimensional imaging method that allows the generation of a three-dimensional numerical model that represents an observed three-dimensional scene. The generated model optimally combines (404) data from two intermediate three-dimensional numerical models, respectively obtained by a stereoscopic three-dimensional reconstruction calculation method (402) and by a method of three-dimensional reconstruction calculation through learning (403). In addition, each new model that is generated helps to improve the overall performance of the method of three-dimensional reconstruction calculation by learning. ## Claims ### Claim 1 A dynamic three-dimensional imaging method comprising the following steps, executed by a calculating unit: receiving at least two two-dimensional digital images representing an observed three-dimensional scene, acquired by at least one determined image sensor from at least two respective positions in space; generating, from the two-dimensional digital images, based on a stereoscopic three-dimensional reconstruction calculation method, a first intermediate three-dimensional numerical model associated with the observed three-dimensional scene; generating, from the two-dimensional digital images, based on a method of three-dimensional reconstruction calculation by learning, a second intermediate three-dimensional numerical model associated with the observed three-dimensional scene; generating, by combination of the first and second intermediate three-dimensional numerical models, a final three-dimensional numerical model, with said generation comprising selecting, for a determined portion of the final three-dimensional numerical model, a corresponding portion of that of the first and second intermediate three-dimensional numerical models which maximizes a quality criterion of the representation of the observed three-dimensional scene based on a comparison of at least one characteristic parameter of said first and second intermediate three-dimensional numerical models for said corresponding portion which is representative of the quality of the representation of the observed three-dimensional scene; storing, in a memory, digital information comprising at least one received two-dimensional digital image, the final three-dimensional numerical model associated with said two-dimensional digital image, and a characteristic parameter of said final three-dimensional numerical model, representative of the quality of the representation of the three-dimensional scene observed; and, using stored digital information as learning data to train and retrain the calculating unit for the execution of the method of three-dimensional reconstruction calculation by learning. ### Claim 2 (depends on 1) The method according to claim 1, further comprising an acquisition, by at least one determined image sensor, of two-dimensional digital images representing the observed three-dimensional scene. ### Claim 3 (depends on 1) The method according to claim 1, further comprising displaying the final three-dimensional numerical model, by a display device. ### Claim 4 (depends on 1) The method according to claim 1, wherein generating the final three-dimensional numerical model by combining the first and second intermediate three-dimensional numerical models comprises: if only the first or the second intermediate three-dimensional numerical model has a value for a voxel modeling a given portion of the representation of the observed three-dimensional scene, then the value is assigned to the final three-dimensional numerical model; if both the first and the second three-dimensional numerical model have a value for a voxel modeling a given portion of the representation of the observed three-dimensional scene, then the value assigned to the final three-dimensional numerical model is selected from among these two respective values based on the comparison of at least one characteristic parameter of said three-dimensional numerical models which is associated with the value of this voxel; and if no three-dimensional numerical model has a value for a voxel modeling a given portion of the representation of the observed three-dimensional scene, then no value is assigned to this voxel or the value assigned to this voxel is determined according to a smoothing calculation method. ### Claim 5 (depends on 1) The method according to claim 1, wherein the generation of the second intermediate three-dimensional numerical model is carried out prior to the generation of the first intermediate three-dimensional numerical model, and wherein the second generated intermediate three-dimensional numerical model is used as an a priori for the generation of the first intermediate three-dimensional numerical model by the stereoscopic three-dimensional reconstruction calculation method. ### Claim 6 (depends on 1) The method according to claim 1, wherein the characteristic parameter of the three-dimensional numerical models is the uncertainty associated with each of the values of said three-dimensional numerical models. ### Claim 7 (depends on 1) The method according to claim 1, wherein the learning data used by the calculating unit to train and retrain the method of three-dimensional reconstruction calculation by learning comprise: data from databases, shared or not, comprising two-dimensional digital images and associated three-dimensional numerical models; and/or data comprising two-dimensional digital images and associated three-dimensional numerical models resulting from the execution of the method by another calculating unit; and data comprising two-dimensional digital images and associated three-dimensional numerical models resulting from the execution of another method by a calculating unit. ### Claim 8 (depends on 1) The method according to claim 1, wherein each intermediate three-dimensional numerical model is either a depth map or a three-dimensional point cloud. ### Claim 9 (depends on 1) The method according to claim 1, wherein, during one or more steps of generating intermediate three-dimensional numerical models, a second remote calculating unit cooperates with the calculating unit to generate said one or more intermediate three-dimensional numerical models. ### Claim 10 (depends on 1) The method according to claim 1, wherein the received two-dimensional digital images are color digital images and wherein the final generated three-dimensional numerical model comprises textures determined based on the colors of said two-dimensional digital images. ### Claim 11 (depends on 1) The method according to claim 1, wherein, during one or more steps of generating intermediate three-dimensional numerical models, the three-dimensional reconstruction calculations carried out use a foveated imaging-type calculation model to obtain an intermediate three-dimensional numerical model which is relatively more precise on the part of said model which projects onto the center of the image and relatively less precise on the part of said model which projects onto the periphery of the image. ### Claim 12 A calculating unit, comprising hardware and software means for executing the steps of: receiving at least two two-dimensional digital images representing an observed three-dimensional scene, acquired by at least one determined image sensor from at least two respective positions in space; generating, from the two-dimensional digital images, based on a stereoscopic three-dimensional reconstruction calculation method, a first intermediate three-dimensional numerical model associated with the observed three-dimensional scene; generating, from the two-dimensional digital images, based on a method of three-dimensional reconstruction calculation by learning, a second intermediate three-dimensional numerical model associated with the observed three-dimensional scene; generating, by combination of the first and second intermediate three-dimensional numerical models, a final three-dimensional numerical model, with said generation comprising selecting, for a determined portion of the final three-dimensional numerical model, a corresponding portion of that of the first and second intermediate three-dimensional numerical models which maximizes a quality criterion of the representation of the observed three-dimensional scene based on a comparison of at least one characteristic parameter of said first and second intermediate three-dimensional numerical models for said corresponding portion which is representative of the quality of the representation of the observed three-dimensional scene; storing, in a memory, digital information comprising at least one received two-dimensional digital image, the final three-dimensional numerical model associated with said two-dimensional digital image, and a characteristic parameter of said final three-dimensional numerical model, representative of the quality of the representation of the three-dimensional scene observed; and, using stored digital information as learning data to train and retrain the calculating unit for the execution of the method of three-dimensional reconstruction calculation by learning. ### Claim 13 A three-dimensional imaging device comprising at least one digital image sensor, a memory and a calculating unit comprising hardware and software means to perform: receiving at least two two-dimensional digital images representing an observed three-dimensional scene, acquired by at least one determined image sensor from at least two respective positions in space; generating, from the two-dimensional digital images, based on a stereoscopic three-dimensional reconstruction calculation method, a first intermediate three-dimensional numerical model associated with the observed three-dimensional scene; generating, from the two-dimensional digital images, based on a method of three-dimensional reconstruction calculation by learning, a second intermediate three-dimensional numerical model associated with the observed three-dimensional scene; generating, by combination of the first and second intermediate three-dimensional numerical models, a final three-dimensional numerical model, with said generation comprising selecting, for a determined portion of the final three-dimensional numerical model, a corresponding portion of that of the first and second intermediate three-dimensional numerical models which maximizes a quality criterion of the representation of the observed three-dimensional scene based on a comparison of at least one characteristic parameter of said first and second intermediate three-dimensional numerical models for said corresponding portion which is representative of the quality of the representation of the observed three-dimensional scene; storing, in a memory, digital information comprising at least one received two-dimensional digital image, the final three-dimensional numerical model associated with said two-dimensional digital image, and a characteristic parameter of said final three-dimensional numerical model, representative of the quality of the representation of the three-dimensional scene observed; and, using stored digital information as learning data to train and retrain the calculating unit for the execution of the method of three-dimensional reconstruction calculation by learning.