# US 6,556,722 B1 · Position determination > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 6,556,722 B1 - Document ID: US6556722B1 - Status: Granted - Kind: B1 - Application number: 09/449,634 - Filed: 1999-11-30 - Granted: 2003-04-29 - Earliest claimed: 1997-05-30 - Assignees: British Broadcasting Corporation - Inventors: Richard Thomas Russell; Graham Alexander Thomas - Web page: https://patentgrep.com/patent/US6556722B1 ## CPC - H04N 5/2224: {related to virtual studio applications} - G01S 5/16: using electromagnetic waves other than radio waves - G01S 5/163: {Determination of attitude (using inertial means G01C9/00; control of attitude G05D1/49)} - G01S 17/42: Simultaneous measurement of distance and other co-ordinates (indirect measurement G01S17/46) - G01S 17/87: Combinations of systems using electromagnetic waves other than radio waves - G01S 17/875: for determining attitude - H04N 5/222: Studio circuitry; Studio devices; Studio equipment (cameras or camera modules comprising electronic image sensors, or control thereof H04N23/00) - H04N 5/272: Means for inserting a foreground image in a background image, i.e. inlay, outlay - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES ## Abstract In the preferred implementation, the position of an object, for example a studio camera is determined by means of a camera which views several markers disposed about a studio ceiling, the markers being patterened as a series of light and dark rings to encode information in binary form enabling the markers to be identified as the camera moves about the studio. Methods and apparatus of more general applicability are also disclosed. ## Claims ### Claim 1 A method of determining the three-dimensional position of an object in a reference frame, the method comprising: providing three or more markers at respective reference positions, at least three of the markers being patterned as a series of concentric shapes encoding a plurality of bits of identification information in the darkness or color of the shapes, the center of the concentric shapes of each patterned marker providing a reference point for each marker, wherein each marker has a respective distance from a reference plane of the reference frame, the distances varying between the markers; storing information including a measure of the three-dimensional positions of the markers in the reference frame and information identifying the patterned markers; obtaining an image of at least a sub-set of said markers from a camera associated with the object; processing the image to identify the positions of said markers in the image and, for each patterned marker in the sub-set, identifying the respective reference point and decoding said identification information; determining a measure of the three-dimensional position of the object in the reference frame based on said processing and decoding and based on said stored information. ### Claim 2 (depends on 1) A method according to claim 1, wherein the patterned markers encode a plurality of bits of information in binary form as a series of light and dark shapes. ### Claim 3 A method of determining the three-dimensional position of an object in a reference frame, the method comprising: providing three or more markers at respective reference positions, at least three of the markers being patterned to encode a plurality of bits of identification information in binary form in a series of concentric light and dark regions, the center of the concentric regions providing a reference point, wherein each marker has a respective distance from a reference plane of the reference frame, the distances varying between the markers;; storing information including a measure of the three-dimensional positions of the markers in the reference frame and information identifying the patterned markers; obtaining an image of at least a sub-set of said plurality of markers from a camera associated with the object; processing the image to identify the positions of said markers in the image and, for each patterned marker in the sub-set, decoding said identification information and identifying the reference point; and determining a measure of the three-dimensional position of the object in the reference frame based on said processing and decoding and based on said stored information. ### Claim 4 (depends on 3) A method of producing a set of at least three markers for use in determining the three-dimensional position of an object in a reference frame in a method according to claim 3, the markers having a mutually different identifiers encoded as a plurality of bits of information, the method comprising: providing mutually different identifiers to be encoded in binary form; and providing on a mountable backing a series of concentric shapes, each shape having either a high or a low brightness or reflectivity, the thickness or brightness or reflectivity of each ring shape selected according to a pre-determined coding scheme to encode the information, wherein the center of the shapes provides a reference point. ### Claim 5 (depends on 2) A method according to claim 2, wherein the shapes are circular rings. ### Claim 6 (depends on 2) A method according to claim 2, wherein 3 or more bits of identification information are encoded. ### Claim 7 (depends on 2) A method according to claim 2, wherein 4 or more shapes are provided. ### Claim 8 (depends on 2) A method according to claim 2, wherein each bit of the information is assigned to a corresponding shape. ### Claim 9 (depends on 2) A method according to claim 2, wherein at least one outer region of a predetermined brightness is provided. ### Claim 10 (depends on 2) A method according to claim 2, wherein at least one parity or other error-correcting bit is included in the encoded information. ### Claim 11 (depends on 2) A method according to claim 2, wherein dark shapes are substantially black. ### Claim 12 (depends on 2) A method according to claim 2, wherein light shapes are retro-reflective so as to reflect light directed from adjacent said camera substantially towards said camera. ### Claim 13 (depends on 12) A method according to claim 12, further including illuminating the markers with a light source substantially coincident with the axis of said camera means so that light falling on said light shapes is retro-reflected towards said camera. ### Claim 14 (depends on 1) A method according to claim 1, wherein each shape is coloured, patterned or shaded to encode a plurality of bits of identification information. ### Claim 15 (depends on 1) A method according to claim 1, wherein all markers are patterned to encode identification information. ### Claim 16 (depends on 1) A method according to claim 1, wherein at least some markers encode identification information uniquely identifying that marker among the plurality of markers. ### Claim 17 (depends on 1) A method according to claim 1, wherein the markers are so disposed that the position of the object can be uniquely determined at all points within a given volume of the reference frame. ### Claim 18 (depends on 1) A method according to claim 1 including adjusting the determined position to smooth out sudden changes in determined position as markers are revealed or obscured. ### Claim 19 (depends on 18) A method according to claim 18, wherein adjusting comprises storing pre-stored or measured marker positions as initial marker positions; dynamically applying correction factors to said initial marker positions to produce respective corrected marker positions; determining the object position based on the corrected marker positions, wherein said dynamically applying correction factors comprises calculating correction factors so that the corrected marker positions are corrected to become mutually self-consistent values, the rate of variation of the correction factors being limited to a pre-determined level by limiting the rate of change of correction factors. ### Claim 20 A method of determining the position of an object based on identification of a plurality of markers at respective reference positions, the method comprising: storing pre-stored or measured marker positions as initial marker positions; dynamically applying correction factors to said initial marker positions to produce respective corrected marker positions; determining the object position based on the corrected marker positions, wherein said dynamically applying correction factors comprises calculating correction factors so that the corrected marker positions are corrected to become mutually self-consistent values, a rate of variation of the correction factors being limited to a pre-determined level by limiting the rate of change of correction factors. ### Claim 21 (depends on 20) A method according to claim 20, wherein the pre-determined level is selected so that transitions in the determined image position resulting from one or more identified markers becoming unusable or new markers becoming usable are smoothed over a period of at least about 2 seconds. ### Claim 22 (depends on 20) A method according to claim 20, wherein a measure of the accuracy of position determination is obtained. ### Claim 23 (depends on 22) A method according to claim 22, wherein said measure is based on determination of said correction factors. ### Claim 24 (depends on 1) A method according to claim 1, wherein said object is a video camera having a field of view differing from the field of view of said camera. ### Claim 25 (depends on 24) A method according to claim 24 further comprising providing at least one supplementary marker at a given position so as to be viewable by the video camera, the or each marker having identification information encoded thereon in binary form as a series of concentric rings of two tones, the tones being selected to enable the marker to be keyed out of the video camera image by chroma-keying, said determining the position of the video camera being based on the position of the supplementary marker, if present, in the video camera image. ### Claim 26 Apparatus for determination of the three-dimensional position of an object in a reference frame, the apparatus comprising: a camera for mounting on an object whose position is to be determined; a memory arranged to store measures of the three-dimensional positions of at least three markers in the reference frame and to store information identifying patterned encoded markers which markers comprise a series of concentric shapes encoding identification information as the darkness or color of the shapes, the center of the concentric shapes of each patterned marker providing a reference point for each marker; image processing means arranged to process an image output by the camera to identify the positions of markers in the image, to decode information encoded in the patterned markers and to determine the position of said reference points; and position determining means arranged to determine the three-dimensional position of the object in the reference frame based on the output of the image processing means and the information stored in the memory means. ### Claim 27 (depends on 26) Apparatus according to claim 26 wherein the memory, image processing means and position determining means are integrated into a single computer. ### Claim 28 (depends on 26) Apparatus according to claim 26 further including a set of at least three markers, including a subset of at least three patterned markers having identification information encoded as a series of concentric rings of light and dark regions, the center of the rings providing a reference point. ### Claim 29 (depends on 26) Apparatus according to claim 26 for use with markers including retro-reflective portions so as to reflect light directed from adjacent said camera substantially towards said camera, the apparatus having light-projection means substantially optically coincident with the axis of said camera. ### Claim 30 (depends on 29) Apparatus according to claim 29 wherein the light-projection means is pulsed, arranged to be pulsed on at a time when said camera means is integrating. ### Claim 31 (depends on 29) Apparatus according to claim 29 wherein the camera means is provided with filter means having a transmission window selected to pass light transmitted by the light-projection means but to inhibit at least some other light. ### Claim 32 (depends on 31) Apparatus according to claim 31, wherein the light-projection means comprises n infra-red LED and said filter inhibits visible light but passes infra-red. ### Claim 33 (depends on 26) Apparatus according to claim 26 including means for applying correction factors to stored or measured marker positions to produce respective corrected marker positions and determining the object position based on the corrected marker positions, the means for applying correction factors comprising: further memory for storing pre-stored or measured marker positions as initial marker positions; means for dynamically applying correction factors to said initial marker positions to produce respective corrected marker positions; means for determining the object position based on the corrected marker positions, wherein said means for dynamically applying correction factors comprises means for calculating correction factors so that the corrected marker positions are corrected to become mutually self-consistent values, the rate of variation of the correction factors being limited to a pre-determined level by limiting the rate of change of correction factors. ### Claim 34 (depends on 26) Apparatus according to claim 26 arranged to determine the position of a video camera wherein said camera comprises a further camera mounted on the video camera, said camera facing in a different direction to the video camera. ### Claim 35 (depends on 34) Apparatus according to claim 34, including means for detecting the zoom and/or focus settings of the video camera. ### Claim 36 (depends on 1) A set of at least three patterned markers mountable in a reference frame to enable the three-dimensional position of an object with respect to the reference frame to be determined in accordance with the method of claim 1, each marker comprising a series of concentric rings, each marker encoding an identifier of the marker as a plurality of bits of information in binary form as the brightness or reflectivity and/or thickness of each ring, the identifier of the markers varying between the markers, the center of the rings providing a reference point for the markers. ### Claim 37 Marker apparatus for positioning in the field of view of a video camera for use in determining the relative position of the camera and the marker apparatus, the apparatus comprising: a base; a first patterned marker mounted at a first predetermined relative positions on a substantially rigid base, the first patterned marker having identification information encoded thereon in binary form as a series of concentric rings of two tones, the tones being selected to enable the marker to be keyed out of the video camera image by chroma-keying, the center of the rings providing a reference point for the marker, the first marker being at a first height from the base; a second patterned marker mounted at a second predetermined relative positions on a substantially rigid base, the second patterned marker having identification information encoded thereon in binary form as a series of concentric rings of two tones, the tones being selected to enable the marker to be keyed out of the video camera image by chroma-keying, the center of the rings providing a reference point for the marker, the second marker being at a second height from the base; a third patterned marker mounted at a third predetermined relative positions on a substantially rigid base, the third patterned marker having identification information encoded thereon in binary form as a series of concentric rings of two tones, the tones being selected to enable the marker to be keyed out of the video camera image by chroma-keying, the center of the rings providing a reference point for the marker, the third marker being at a third height from the base; wherein said first height, said second height and said third height are all mutually different, whereby a three dimensional position fix relative to the base can be obtained from the markers. ### Claim 38 A method of determining the relative three-dimensional positions of at least one camera and an object having at least three markers mounted thereon, the relative positions of the markers and the object being substantially fixed, each marker being patterned to encode identification information in binary form as the darkness or color of a series of concentric rings, the center of the rings providing a reference point, the method comprising: storing the relative three dimensional positions of the markers with respect to the object and information enabling the identification information to be decoded; receiving an image from the camera; identifying the positions of the markers in the camera image; decoding the identification information encoded on each marker; determining the position of the reference point for each marker; and based on the stored information, determining a measure of the relative three-dimensional positions and orientations of said at least one camera and object. ### Claim 39 (depends on 3) A method according to claim 3, wherein the reference frame comprises a production studio having at least 10 markers positioned on the ceiling thereof, wherein the camera is an upward-pointing camera mounted on a studio camera. ### Claim 40 (depends on 1) A virtual production studio having at least three patterned markers mounted on the ceiling thereof at mutually different distances from the ceiling to enable the three-dimensional position of an object with respect to the studio reference frame to be determined in accordance with the method of claim 1, each marker comprising a series of concentric rings, each marker encoding an identifier of the marker as a plurality of bits of information in binary form as the brightness or reflectivity and/or thickness of each ring, the identifier of the markers varying between the markers, the center of the rings providing a reference point for the markers. ### Claim 41 (depends on 40) A virtual production studio according to claim 40, having at least 10 markers mounted on the ceiling thereof. ### Claim 42 (depends on 40) A virtual production studio according to claim 40 including a store of the three-dimensional positions and identifiers of the markers for use in position determination. ### Claim 43 (depends on 40) A virtual production studio according to claim 40 including at least one studio camera arranged to capture a subject in a working space of the studio, the studio camera having apparatus for determination of the three-dimensional position of the studio camera in the studio reference frame, the apparatus for determination of the three-dimensional position comprising: a camera for mounting on the studio camera and arranged to point upwards at the markers when the studio camera is upright and pointed at a subject in said working space; a memory storing measures of the three-dimensional positions of said markers in the reference frame and to store information identifying patterned encoded markers which markers comprise a series of concentric shapes encoding identification information as the darkness or color of shapes, the center of the concentric shapes of each patterned marker providing a reference point for each marker; image processing means arranged to process an image output by the camera to identify the positions of markers in the image, to decode information encoded in the patterned markers and to determine the position of said reference points; and position determining means arranged to determine the three-dimensional position of the studio camera in the studio reference frame based on the output of the image processing means and the information stored in the memory means. ### Claim 44 (depends on 24) A method of providing a virtual production, the method comprising determining the position of a video camera in accordance with claim 24; and applying a video effect to the output of the video camera based on the determined position.