# US 7,613,361 B2 · Information processing method and device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 7,613,361 B2 - Document ID: US7613361B2 - Status: Granted - Kind: B2 - Application number: 11/127,000 - Filed: 2005-05-11 - Published: 2005-11-17 - Granted: 2009-11-03 - Earliest claimed: 2004-05-14 - Assignees: Canon Kabushiki Kaisha - Inventors: Mahoro Anabuki; Kiyohide Satoh; Shinichi Aratani; Daisuke Kotake - Web page: https://patentgrep.com/patent/US7613361B2 ## CPC - G06T 7/246: using feature-based methods, e.g. the tracking of corners or segments - G06T 7/80: Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration - G06T 2207/30241: Trajectory - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract An information processing device for obtaining placement information regarding an imaging device of markers to be calibrated that are provided upon the imaging device, includes: a first image obtaining unit adapted to obtain a first image photographed with the imaging device; a second image obtaining unit adapted to obtain a second image obtained by photographing the imaging device from a bird's-eye view position; a first detecting unit adapted to detect information relating to image coordinates of reference markers placed in a scene, from the first image; a second detecting unit adapted to detect information relating to image coordinates of markers to be calibrated from the second image; and a calibration information calculating unit adapted to obtain the calibration information using information relating to image coordinates of the reference markers detected by the first detecting unit and the second detecting unit. ## Claims ### Claim 1 An information processing method for calculating a position of a marker provided upon an object in an object coordinate system using the object as a reference, the information processing method comprising: an object position-and-orientation information obtaining step for obtaining position and orientation information of the object; a first image input step for inputting a first image from a bird's-eye view imaging unit, photographed from a bird's-eye view of the object in a world coordinate system; a detecting step for detecting the marker from the first image and obtaining a real value of an image coordinate of the detected marker; an image coordinate calculating step for calculating an ideal value of the image coordinate of the marker based on the position and orientation information of the object in the world coordinate system and a state vector representing the position of the marker in the object coordinate system; a correction value calculating step for calculating a correction value of the state vector based on a difference between the real value and the ideal value of the image coordinate of the marker; and a marker position calculating step for calculating the position of the marker in the object coordinate system by correcting the state vector representing the position of the marker in the object coordinate system with the correction value. ### Claim 2 (depends on 1) An information processing method according to claim 1, wherein the object has a first-person view photographing unit; and wherein, the object position-and-orientation information obtaining step further comprises: obtaining a second image photographed with the first-person view photographing unit, extracting markers existing on other than the object from the second image, and obtaining the position and orientation of the first-person view photographing unit based on information relating to the image coordinates of markers extracted from the second image and position information of markers held beforehand. ### Claim 3 (depends on 1) An information processing method according to claim 1, wherein relative positions of a plurality of markers are known. ### Claim 4 (depends on 1) An information processing method according to claim 1, wherein positions of the markers in a marker coordinate system are known; and wherein a conversion parameter is calculated in the marker position calculating step for converting between the coordinate system of the position and orientation information for the object and the marker coordinate system. ### Claim 5 (depends on 1) An information processing method according to claim 1, wherein the relative position and orientation of the object and the bird's-eye view photographing unit are taken as unknown parameters. ### Claim 6 (depends on 5) An information processing method according to claim 5, further comprising calculating of position and orientation of the bird's-eye view photographing unit as to the object. ### Claim 7 (depends on 1) An information processing method according to claim 1, wherein the bird's-eye view photographing unit is fixed. ### Claim 8 (depends on 1) An information processing method according to claim 1, wherein the marker position calculating step further comprises: an estimating step for estimating information relating to image coordinates of the markers in the image based on estimation values of position of the markers with regard to the object; and a correcting step for correcting the estimation values based on the error between information relating to image coordinates of the markers detected in the detecting step and information relating to image coordinates of the markers estimated in the estimating step. ### Claim 9 (depends on 1) An information processing method according to claim 1, wherein the marker position calculating step calculates the position and orientation of the marker with the object as a reference. ### Claim 10 (depends on 9) An information processing method according to claim 9, wherein the marker position calculating step further comprises: an estimating step for estimating information relating to image coordinates of the markers in the image based on estimation values of position and orientation of the markers with regard to the object; and a correcting step for correcting the estimation values based on the error between information relating to image coordinates of the markers detected in the detecting step and information relating to image coordinates of the markers estimated in the estimating step. ### Claim 11 An information processing device for calculating a position of a marker provided upon an object in an object coordinate system using the object as a reference, the device comprising: an object position-and-orientation obtaining unit adapted to obtain a position and orientation of the object; an image input unit adapted to input an image from a bird's-eye view of the object; a detecting unit adapted to detect the marker from the image and obtain a real value of an image coordinate of the detected marker; an image coordinate calculating unit adapted to calculate an ideal value of the image coordinate of the marker based on the position and orientation information of the object in the world coordinate system and a state vector representing the position of the marker in the object coordinate system; a correction value calculating unit adapted to calculate a correction value of the state vector based on a difference between the real value and the ideal value of the image coordinate of the marker; and a marker position calculating unit adapted to calculate the position of the marker with in the object coordinate system by correcting the state vector representing the position of the marker in the object coordinate system with the correction value. ### Claim 12 (depends on 11) An information processing device according to claim 11, wherein the marker position calculating unit is adapted to calculate the position and orientation of the marker with the object as a reference. ### Claim 13 A computer-storage medium having a computer-executable program stored thereon for executing an information processing method for calculating a position of a marker provided upon an object, as to the object, the information processing method comprising: an object position-and-orientation obtaining step for obtaining a position and orientation of the object in a world coordinate system; a first image input step for inputting a first image from a bird's-eye view imaging unit, photographed from a bird's-eye view of the object; a detecting step for detecting the marker from the first image and obtaining a real value of an image coordinate of the detected marker; an image coordinate calculating step for calculating an ideal value of the image coordinate of the marker based on the position and orientation information of the object in the world coordinate system and a state vector representing the position of the marker in the object coordinate system; a correction value calculating step for calculating a correction value of the state vector based on a difference between the real value and the ideal value of the image coordinate of the marker; and a marker position calculating step for calculating the position of the marker in the object coordinate system by correcting the state vector representing the position of the marker in the object coordinate system with the correction value.