# US 2019/0318201 A1 · Methods and systems for shape based training for an object detection algorithm > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0318201 A1 - Document ID: US20190318201A1 - Status: Published application - Kind: A1 - Application number: 15/951,840 - Filed: 2018-04-12 - Published: 2019-10-17 - Earliest claimed: 2018-04-12 - Assignees: Seiko Epson Corporation - Applicants: Seiko Epson Corporation - Inventors: Mirza Tahir AHMED; Dibyendu MUKHERJEE; Jie WANG - Web page: https://patentgrep.com/patent/US20190318201A1 ## CPC - G06T 17/00: Three-dimensional [3D] modelling for computer graphics - G06F 18/2148: {characterised by the process organisation or structure, e.g. boosting cascade} - G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts - G06V 10/255: {Detecting or recognising potential candidate objects based on visual cues, e.g. shapes} - G06V 10/774: Generating sets of training patterns; Bootstrap methods, e.g. bagging or boosting - G06V 20/64: Three-dimensional [3D] objects - G06V 20/653: {by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces} - G06T 2219/20: Indexing scheme for editing of 3D models - G06T 2219/2021: Shape modification - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING ## Abstract A non-transitory computer readable medium embodies instructions that cause one or more processors to perform a method. The method includes: (A) receiving, in one or more memories, a 3D model corresponding to an object, and (B) setting a depth sensor characteristic data set for a depth sensor for use in detecting a pose of the object in a real scene. The method also includes (C) generating blurred 2.5D representation data of the 3D model for at least one view around the 3D model based on the 3D model and the depth sensor characteristic data set, to generate, on the basis of the 2.5D representation data, training data for training an object detection algorithm, and (D) storing the training data in one or more memories. ## Claims ### Claim 1 A non-transitory computer readable medium that embodies instructions that, when executed by one or more processors, cause the processors to perform a method comprising: (A) receiving, in one or more memories, a 3D model corresponding to an object; (B) setting a depth sensor characteristic data set for a depth sensor for use in detecting a pose of the object in a real scene; (C) generating blurred 2.5D representation data of the 3D model for at least one view around the 3D model based on the 3D model and the depth sensor characteristic data set, to generate, on the basis of the 2.5D representation data, training data for training an object detection algorithm; and (D) storing the training data in one or more memories. ### Claim 2 (depends on 1) A non-transitory computer readable medium according to claim 1, wherein (C) generating blurred 2.5D representation data of the 3D model comprises: blurring the 3D model based on the depth sensor characteristic data set; and extracting 2.5D representation data of the blurred 3D model for at least one view around the 3D model to generate the blurred 2.5D representation data. ### Claim 3 (depends on 1) A non-transitory computer readable medium according to claim 1, wherein (C) generating blurred 2.5D representation data of the 3D model comprises: extracting 2.5D representation data of the 3D model for at least one view around the 3D model; blurring the extracted 2.5D representation data of the 3D model based on the depth sensor characteristic data set to generate the blurred 2.5D representation data. ### Claim 4 (depends on 1) A non-transitory computer readable medium according to claim 1, further comprising: identifying at least one edge and/or one surface feature of the blurred 2.5D representation data of the 3D model; storing the identified at least one edge and/or at least one surface feature as at least a portion of the training data. ### Claim 5 (depends on 1) A non-transitory computer readable medium according to claim 1, wherein (B) setting the depth sensor characteristic data set includes: receiving information identifying an object detection device having the depth sensor, acquiring, based at least in part on the information identifying the object detection device, the depth sensor characteristic data set from a plurality of the depth sensor characteristic data sets stored in one or more memories, wherein each depth sensor characteristic data set of the plurality of depth sensor characteristic data sets is associated in the one or more memories with at least one object detection device of a plurality of different object detection devices. ### Claim 6 (depends on 1) A non-transitory computer readable medium according to claim 1, wherein (B) setting the depth sensor characteristic data set includes: acquiring, through a data connection, the depth sensor characteristic data set from an object detection device having the depth sensor when the object detection device becomes accessible by the one or more processors through the data connection. ### Claim 7 (depends on 1) A non-transitory computer readable medium according to claim 1, wherein (C) generating blurred 2.5D representation data of the 3D model comprises: generating blurred 2.5D representation data of the blurred 3D model for views distributed over a range of 360 degrees of azimuth and/or elevation around the 3D model based on the 3D model and the depth sensor characteristic data set. ### Claim 8 A method comprising: (A) receiving, in one or more memories, a 3D model corresponding to an object; (B) setting a depth sensor characteristic data set for a depth sensor for use in detecting a pose of the object in a real scene; (C) generating blurred 2.5D representation data of the 3D model for at least one view around the 3D model based on the 3D model and the depth sensor characteristic data set, to generate, on the basis of the 2.5D representation data, training data for training an object detection algorithm; and (D) storing the training data in one or more memories. ### Claim 9 (depends on 8) A method according to claim 8, wherein (C) generating blurred 2.5D representation data of the 3D model comprises: blurring the 3D model based on the depth sensor characteristic data set; and extracting 2.5D representation data of the blurred 3D model for at least one view around the 3D model to generate the blurred 2.5D representation data. ### Claim 10 (depends on 8) A method according to claim 8, wherein (C) generating blurred 2.5D representation data of the 3D model comprises: extracting 2.5D representation data of the 3D model for at least one view around the 3D model; blurring the extracted 2.5D representation data of the 3D model based on the depth sensor characteristic data set to generate the blurred 2.5D representation data. ### Claim 11 (depends on 8) A method according to claim 8, further comprising: identifying at least one edge of the blurred 2.5D representation data of the 3D model; identifying at least one surface feature of the blurred 2.5D representation data of the 3D model; and storing the identified at least one edge and at least one surface feature as at least a portion of the training data. ### Claim 12 (depends on 8) A method according to claim 8, wherein (B) setting the depth sensor characteristic data set includes: receiving information identifying an object detection device having the depth sensor, acquiring, based at least in part on the information identifying the object detection device, the depth sensor characteristic data set from a plurality of the depth sensor characteristic data sets stored in one or more memories, wherein each depth sensor characteristic data set of the plurality of depth sensor characteristic data sets is associated in the one or more memories with at least one object detection device of a plurality of different object detection devices. ### Claim 13 (depends on 8) A method according to claim 8, wherein (B) setting the depth sensor characteristic data set includes: acquiring, through a data connection, the depth sensor characteristic data set from an object detection device having the depth sensor when the object detection device becomes accessible by the one or more processors through the data connection. ### Claim 14 (depends on 8) A method according to claim 8, wherein (C) generating blurred 2.5D representation data of the 3D model comprises: generating blurred 2.5D representation data of the blurred 3D model for views distributed over a range of 360 degrees of azimuth and/or elevation around the 3D model based on the 3D model and the depth sensor characteristic data set. ### Claim 15 A training apparatus comprising: one or more memories; and a processor communicatively coupled to the one or more memories, the processor configured by instructions stored in the one or more memories to perform a method comprising: (A) receiving, in the one or more memories, a 3D model corresponding to an object; (B) setting a depth sensor characteristic data set for a depth sensor for use in detecting a pose of the object in a real scene; (C) generating blurred 2.5D representation data of the 3D model for at least one view around the 3D model based on the 3D model and the depth sensor characteristic data set, to generate, on the basis of the 2.5 representation data, training data for training an object detection algorithm; and (D) storing the training data in the one or more memories. ### Claim 16 (depends on 15) A training apparatus according to claim 15, wherein (C) generating blurred 2.5D representation data of the 3D model comprises: blurring the 3D model based on the depth sensor characteristic data set; and extracting 2.5D representation data of the blurred 3D model for at least one view around the 3D model to generate the blurred 2.5D representation data. ### Claim 17 (depends on 15) A training apparatus according to claim 15, wherein (C) generating blurred 2.5D representation data of the 3D model comprises: extracting 2.5D representation data of the 3D model for at least one view around the 3D model; blurring the extracted 2.5D representation data of the 3D model based on the depth sensor characteristic data set to generate the blurred 2.5D representation data. ### Claim 18 (depends on 15) A training apparatus according to claim 15, further comprising: identifying at least one edge of the blurred 2.5D representation data of the 3D model; identifying at least one surface feature of the blurred 2.5D representation data of the 3D model; and storing the identified at least one edge and at least one surface feature as at least a portion of the training data. ### Claim 19 (depends on 15) A training apparatus according to claim 15, wherein (B) setting the depth sensor characteristic data set includes: receiving information identifying an object detection device having the depth sensor, acquiring, based at least in part on the information identifying the object detection device, the depth sensor characteristic data set from a plurality of the depth sensor characteristic data sets stored in one or more memories, wherein each depth sensor characteristic data set of the plurality of depth sensor characteristic data sets is associated in the one or more memories with at least one object detection device of a plurality of different object detection devices. ### Claim 20 (depends on 15) A training apparatus according to claim 15, wherein (B) setting the depth sensor characteristic data set includes: acquiring, through a data connection, the depth sensor characteristic data set from an object detection device having the depth sensor when the object detection device becomes accessible by the one or more processors through the data connection.