# US 2019/0172226 A1 · System and method for generating training images > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0172226 A1 - Document ID: US20190172226A1 - Status: Published application - Kind: A1 - Application number: 16/211,062 - Filed: 2018-12-05 - Published: 2019-06-06 - Earliest claimed: 2017-12-05 - Assignees: Shortbite Ltd. - Applicants: Shortbite Ltd. - Inventors: Martin Paul Fergie; David William Higham - Web page: https://patentgrep.com/patent/US20190172226A1 ## CPC - G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration - G06F 7/58: Random or pseudo-random number generators - G06T 3/067: Reshaping or unfolding three-dimensional [3D] tree structures onto two-dimensional [2D] planes - G06T 3/12: Panospheric to cylindrical image transformations - G06T 3/608: by skew deformation, e.g. two-pass or three-pass rotation - G06T 5/20: using local operators - G06T 5/80: Geometric correction - G06T 7/13: Edge detection - G06T 2207/10004: Still image; Photographic image - G06T 2207/20081: Training; Learning - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract A computer implemented system and method for automatically generating a training set of image comprises obtaining an image in which each pixel represents a 3 dimensional ‘3D’ point on the surface of a sphere, obtaining a projective camera model comprising a projection matrix which defines a projective transformation to map 3D points on the surface of a sphere onto an image plane based on camera model parameter values, applying the projective camera model with selected camera model parameter values to the image to produce a rectilinear image; storing the rectilinear image in association with the selected camera model parameter values; and repeating the applying and storing for different selected camera model parameter values. ## Claims ### Claim 1 A computer implemented system for automatically generating a training set of images, the system comprising: first obtaining means arranged to obtain an image in which each pixel represents a 3 dimensional ‘3D’ point on the surface of a sphere; second obtaining means arranged to obtain a projective camera model comprising a projection matrix which defines a projective transformation to map 3D points on the surface of a sphere onto an image plane based on camera model parameter values; applying means arranged to apply the projective camera model with selected camera model parameter values to the image to produce a rectilinear image; storing means arranged to store the rectilinear image in association with the selected camera model parameter values; and the system being arranged to repeat the applying and storing for different selected camera model parameter values. ### Claim 2 (depends on 2) The system according to claim 2, wherein the first obtaining means is further arranged to: obtain an equirectangular image; and transform the equirectangular image into a transformed image in which each pixel represents a 3D point on the surface of a sphere. ### Claim 3 (depends on 1) The system according to claim 1, wherein the second obtaining means is arranged to randomly select the camera model parameter values. ### Claim 4 (depends on 1) The system according to claim 1, wherein the second obtaining means is arranged select the camera model parameter values according to a predetermined sequence. ### Claim 5 (depends on 4) The system according to claim 4, wherein the predetermined sequence is a pseudo-random sequence. ### Claim 6 (depends on 1) The system according to claim 1, wherein the applying means is arranged to apply the projective camera model with a selected orientation. ### Claim 7 (depends on 6) The system according to claim 6, wherein the applying means is arranged to randomly select the selected orientation. ### Claim 8 (depends on 6) The system according to claim 6, wherein the applying means is arranged to select the selected orientation according to a predetermined sequence. ### Claim 9 (depends on 8) The system according to claim 8, wherein the predetermined sequence is a pseudo-random sequence. ### Claim 10 (depends on 1) The system according to claim 1, wherein the camera model parameter values are intrinsic camera parameters. ### Claim 11 (depends on 10) The system according to claim 10, wherein the intrinsic camera parameters comprise one or more of: focal length, pixel skew, principal point, or lens distortion. ### Claim 12 (depends on 1) The system according to claim 1, further comprising filtering means arranged to filter the produced rectilinear image to determine whether to accept the rectilinear image for use as a training image, wherein the storing means stores the rectilinear image in association with the selected camera model parameter values only if the rectilinear image is accepted by the filtering means. ### Claim 13 (depends on 12) The system according to claim 12, wherein the filtering means is arranged to: perform line segment extraction on the rectilinear image; compare the number of line segments identified to a first predetermined threshold; and accept the rectilinear image if the result of the comparison is that the number of line segments exceeds the first predetermined threshold. ### Claim 14 (depends on 1) The system according to claim 1, further comprising image count means arranged to maintain a count of the number of rectilinear images which are derived from an image, wherein the system is arranged to obtain another image when the count reaches a predetermined number. ### Claim 15 (depends on 14) The system according to claim 14, wherein the image count means is arranged to maintain a count of the number of rectilinear images which are derived from an image and stored. ### Claim 16 A computer implemented method of automatically generating a training set of images, the method comprising: obtaining an image in which each pixel represents a 3 dimensional ‘3D’ point on the surface of a sphere; obtaining a projective camera model comprising a projection matrix which defines a projective transformation to map 3D points on the surface of a sphere onto an image plane based on camera model parameter values; applying the projective camera model with selected camera model parameter values to the image to produce a rectilinear image; storing the rectilinear image in association with the selected camera model parameter values; and repeating the applying and storing for different selected camera model parameter values. ### Claim 17 (depends on 16) The method according to claim 16, wherein the obtaining an image comprises: obtaining an equirectangular image; and transforming the equirectangular image into a transformed image in which each pixel represents a 3D point on the surface of a sphere. ### Claim 18 (depends on 16) The method according to claim 16, wherein the selected camera model parameter values are randomly selected. ### Claim 19 (depends on 16) The method according to claim 16, wherein the selected camera model parameter values are selected according to a predetermined sequence. ### Claim 20 (depends on 19) The method according to claim 19, wherein the predetermined sequence is a pseudo-random sequence. ### Claim 21 (depends on 16) The method according to claim 16, wherein the applying the projective camera model is carried out with a selected orientation. ### Claim 22 (depends on 21) The method according to claim 21, wherein the selected orientation is randomly selected. ### Claim 23 (depends on 22) The method according to claim 22, wherein the selected orientation is selected according to a predetermined sequence. ### Claim 24 (depends on 22) The method according to claim 22, wherein the predetermined sequence is a pseudo-random sequence. ### Claim 25 (depends on 16) The method according to claim 16, wherein the camera model parameter values are intrinsic camera parameters. ### Claim 26 (depends on 16) The method according to claim 16, wherein the intrinsic camera parameters comprise one or more of: focal length, pixel skew, principal point, or lens distortion. ### Claim 27 (depends on 16) The method according to claim 16, further comprising filtering the produced rectilinear image to determine whether to accept the rectilinear image for use as a training image, and storing the rectilinear image in association with the selected camera model parameter values only if it is accepted. ### Claim 28 (depends on 27) The method according to claim 27, wherein the filtering comprises: performing line segment extraction on the rectilinear image; comparing the number of line segments identified to a first predetermined threshold; and accepting the rectilinear image if the result of the comparison is that the number of line segments exceeds the first predetermined threshold. ### Claim 29 (depends on 16) The method according to claim 16, further comprising: maintaining a count of the number of rectilinear images which are derived from an image; and when the count reaches a predetermined number, obtaining another image. ### Claim 30 (depends on 29) The method according to claim 29, wherein the maintained count is of the number of rectilinear images which are derived from an image and stored. ### Claim 31 A computer program comprising computer readable instructions which, when executed by a processor of a computer cause the computer to carry out a method comprising: obtaining an image in which each pixel represents a 3 dimensional ‘3D’ point on the surface of a sphere; obtaining a projective camera model comprising a projection matrix which defines a projective transformation to map 3D points on the surface of a sphere onto an image plane based on camera model parameter values; applying the projective camera model with selected camera model parameter values to the image to produce a rectilinear image; storing the rectilinear image in association with the selected camera model parameter values; and repeating the applying and storing for different selected camera model parameter values.