# US 9,039,528 B2 · Visual target tracking > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,039,528 B2 - Document ID: US9039528B2 - Status: Granted - Kind: B2 - Application number: 13/309,306 - Filed: 2011-12-01 - Published: 2012-03-29 - Granted: 2015-05-26 - Earliest claimed: 2009-01-30 - Assignees: Microsoft Technology Licensing, LLC - Applicants: Ryan M. Geiss - Inventors: Ryan M. Geiss - Web page: https://patentgrep.com/patent/US9039528B2 ## CPC - G06T 7/251: {involving models} - A63F 13/213: comprising photodetecting means, e.g. cameras, photodiodes or infrared cells (A63F13/219 takes precedence) - A63F 13/428: involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes - A63F 2300/1093: using visible light - A63F 2300/6607: for animating game characters, e.g. skeleton kinematics - G06T 2207/10021: Stereoscopic video; Stereoscopic image sequence - G06T 2207/10028: Range image; Depth image; 3D point clouds - G06T 2207/30196: Human being; Person - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - A63F: CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR ## Abstract A method of tracking a target includes receiving an observed depth image of the target from a source and obtaining a posed model of the target. The model is rasterized into a synthesized depth image, and the pose of the model is adjusted based, at least in part, on differences between the observed depth image and the synthesized depth image. ## Claims ### Claim 1 A method of electronic gaming, the method comprising; representing a human game player with a machine-readable model configured for adjustment into a plurality of different poses; receiving an observed depth image of the human game player from a source; rasterizing the machine-readable model of the human game player into a synthesized depth image; adjusting a pose of the machine-readable model based, at least in part, on differences between the observed depth image and the synthesized depth image; and using the pose of the machine-readable model, as adjusted, as a parameter to affect an element of game play. ### Claim 2 (depends on 1) The method of claim 1, where the observed depth image includes a plurality of observed pixels, one or more observed pixels including an observed depth value indicating a distance between the source and a portion of the human game player corresponding to that observed pixel. ### Claim 3 (depends on 1) The method of claim 1, where the synthesized depth image includes a plurality of synthesized pixels, one or more synthesized pixels including a synthesized depth value indicating a distance between the source and a portion of the human game player represented by that synthesized pixel. ### Claim 4 (depends on 1) The method of claim 1, where the synthesized depth image includes a plurality of synthesized pixels, one or more synthesized pixels including a body-part index indicating a portion of the human game player to which that synthesized pixel corresponds. ### Claim 5 (depends on 1) The method of claim 1, where adjusting the pose of the machine-readable model is based on a difference between an observed depth value of an observed pixel and a synthesized depth value of a corresponding synthesized pixel. ### Claim 6 A method of tracking a target, the method comprising: receiving an observed depth image of the target from a source; obtaining a model of the target, the model having a pose; rasterizing the model into a synthesized depth image; adjusting the pose of the model based, at least in part, on differences between the observed depth image and the synthesized depth image. ### Claim 7 (depends on 6) The method of claim 6, where the observed depth image includes a plurality of observed pixels, one or more observed pixels including an observed depth value indicating a distance between the source and a portion of the target corresponding to that observed pixel. ### Claim 8 (depends on 6) The method of claim 6, where the source includes a depth camera. ### Claim 9 (depends on 6) The method of claim 6, where the source includes stereo cameras. ### Claim 10 (depends on 6) The method of claim 6, where the model includes a plurality of body parts adjustably connected by a plurality of joints. ### Claim 11 (depends on 6) The method of claim 6, where the model includes a plurality of skeletal members adjustably connected by a plurality of joints. ### Claim 12 (depends on 6) The method of claim 6, where the model includes a wireframe mesh including a plurality of polygons adjustably connected at polygonal edges. ### Claim 13 (depends on 6) The method of claim 6, where the synthesized depth image includes a plurality of synthesized pixels, one or more synthesized pixels including a synthesized depth value indicating a distance between the source and a portion of the target represented by that synthesized pixel. ### Claim 14 (depends on 6) The method of claim 6, where the synthesized depth image includes a plurality of synthesized pixels, one or more synthesized pixels including a body-part index indicating a portion of the target to which that synthesized pixel corresponds. ### Claim 15 (depends on 6) The method of claim 6, where the synthesized depth image includes a plurality of synthesized pixels, each synthesized pixel including a target index indicating a target to which that synthesized pixel corresponds. ### Claim 16 (depends on 6) The method of claim 6, where adjusting the pose of the model is based on a difference between an observed depth value of an observed pixel and a synthesized depth value of a corresponding synthesized pixel. ### Claim 17 (depends on 6) The method of claim 6, where adjusting the pose of the model is further based on a body-part index of an observed pixel that differs from a corresponding synthesized pixel. ### Claim 18 (depends on 6) The method of claim 6, where adjusting the pose of the model includes rectifying the model in accordance with one or more constraints. ### Claim 19 A computing system, comprising: a source configured to capture depth information; a logic subsystem operatively connected to the source; and a data-holding subsystem holding instructions executable by the logic subsystem to: receive an observed depth image of a target from the source; obtain a model of the target, the model having a pose; rasterize the model into a synthesized depth image; adjust the pose of the model based, at least in part, on differences between the observed depth image and the synthesized depth image.