# US 10,771,776 B2 · Apparatus and method for generating a camera model for an imaging system > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,771,776 B2 - Document ID: US10771776B2 - Status: Granted - Kind: B2 - Application number: 16/111,676 - Filed: 2018-08-24 - Published: 2019-03-14 - Granted: 2020-09-08 - Earliest claimed: 2017-09-12 - Assignees: Sony Corporation - Applicants: Sony Corporation - Inventors: Hendrik Schilling - Web page: https://patentgrep.com/patent/US10771776B2 ## CPC - H04N 17/002: {for television cameras} - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration - G06T 15/06: Ray-tracing - G06T 17/20: Finite element generation, e.g. wire-frame surface description, {tesselation} - H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums - G06T 2207/30204: Marker - G06T 2210/21: Collision detection, intersection - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract The disclosure pertains to an apparatus comprising a circuitry. The circuitry is configured to obtain a calibration image of a target, to derive a sparse image based on the calibration image, wherein the sparse image includes image points, to derive ray support points based on the image points by performing an image to target mapping of the image points based on a polynomial function, wherein the ray support points being indicative of light rays reflected by the target and incidenting on an image sensor, and to generate a camera model based on the derived ray support points. ## Claims ### Claim 1 An apparatus comprising: circuitry configured to obtain a calibration image of a target; derive a sparse image based on the calibration image, wherein the sparse image includes image points; derive ray support points based on the image points by performing an image to target mapping of the image points based on a polynomial function, wherein the ray support points are indicative of light rays reflected by the target and incidenting on an image sensor; and generate a camera model based on the derived ray support points; derive an error metric of the derived ray support points, wherein the camera model is generated further based on the error metric; calculate ray-target intersections; compare the ray-target intersections with the ray support points to derive the error metric of the derived ray support points; and transform the error metric from target coordinates into image coordinates. ### Claim 2 (depends on 1) The apparatus of claim 1, wherein the target is a passive target. ### Claim 3 (depends on 1) The apparatus of claim 1, wherein the image to target mapping is performed based on a generic 2D polynomial function. ### Claim 4 (depends on 1) The apparatus of claim 1, wherein the circuitry, is further configured to generate weighted collections of image points, wherein the image to target mapping is performed based on the weighted collections of image points. ### Claim 5 (depends on 1) The apparatus of claim 1, wherein the circuitry is further configured to model a target deformation based on a low resolution mesh whose deformation is applied to the ray support points of the sparse image based on bilinear interpolation. ### Claim 6 (depends on 1) The apparatus of claim 1, wherein the circuitry is further configured to obtain a further calibration image representing a different view on the target than the calibration image; and generate the camera model further based on the further calibration image. ### Claim 7 (depends on 1) The apparatus of claim 1, wherein the calibration image is captured by a camera of a multiple camera device, wherein the circuitry is further configured to obtain a calibration image captured by another camera of the multiple camera device; and generate the camera model further based on the calibration image captured by the other camera, wherein the camera model is a camera model for the multiple camera device. ### Claim 8 (depends on 1) The apparatus of claim 1, further comprising the image sensor, wherein the image sensor comprises a plurality of pixels, wherein the camera model indicates ray support points derived from different calibration images, wherein the indicated ray support points derived from the different calibration images are to be projected onto a corresponding pixel of the image sensor. ### Claim 9 A method, comprising: obtaining a calibration image of a target; deriving a sparse image based on the calibration image, wherein the sparse image includes image points; deriving ray support points based on the image points by performing an image to target mapping of the image points based on a polynomial function, wherein the ray support points being indicative of light rays reflected by the target and incidenting on an image sensor; generating a camera model based on the derived ray support points; deriving, an error metric of the derived ray support points, wherein the camera model is generated further based on the error metric; calculating ray-target intersections; comparing the ray-target intersections with the ray support points to derive the error metric of the derived ray support points; and transforming the error metric from target coordinates into image coordinates. ### Claim 10 (depends on 9) The method of claim 9, wherein the target is a passive target. ### Claim 11 (depends on 9) The method of claim 9, wherein the image to target mapping is performed based on a generic 2D polynomial function. ### Claim 12 (depends on 9) The method of claim 9, further comprising generating weighted collections of image points, wherein the image to target mapping is performed based on the weighted collections of image points. ### Claim 13 (depends on 9) The method of claim 9, further comprising modeling a target deformation based on a low resolution mesh whose deformation is applied to the ray support points of each sparse image based on bilinear interpolation. ### Claim 14 (depends on 9) The method of claim 9, further comprising: obtaining a further calibration image representing a different view on the target than the calibration image; and generating the camera model further based on the further calibration image. ### Claim 15 (depends on 9) The method of claim 9, wherein the calibration image is captured by a camera of a multiple camera device, wherein the method further comprises: obtaining a calibration image captured by another camera of the multiple camera device; and generating the camera model further based on the calibration image captured by the other camera, wherein the camera model is a camera model for the multiple camera device. ### Claim 16 (depends on 9) The method of claim 9, wherein the camera model indicates ray support points derived from different calibration images, wherein the indicated ray support points derived from the different calibration images are to be projected of to a corresponding pixel of the image sensor.