# US 9,520,072 B2 · Systems and methods for projecting images onto an object > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,520,072 B2 - Document ID: US9520072B2 - Status: Granted - Kind: B2 - Application number: 13/624,371 - Filed: 2012-09-21 - Published: 2013-03-21 - Granted: 2016-12-13 - Earliest claimed: 2012-09-21 - Assignees: University of South Florida - Applicants: University of South Florida (A Florida Non-Profit Corporation) - Inventors: Yu Sun; Adrian S. Johnson - Web page: https://patentgrep.com/patent/US9520072B2 ## CPC - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G09B 19/003: {Repetitive work cycles; Sequence of movements} - G09B 23/30: Anatomical models {(G09B23/281 - G09B23/288 take precedence)} - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G09B: EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS (devices for psychotechnics or for testing reaction times A61B5/16; games, sports, amusements A63; projectors, projector screens G03B) ## Abstract In one embodiment, a method for projecting images on a subject includes determining a pose and position of the subject, adjusting a three-dimensional model of an anatomical structure of the subject to match the determined pose and position, and projecting an image of the anatomical structure onto the subject in registration with the actual anatomical structure of the subject to illustrate the location of the structure on or within the subject. ## Claims ### Claim 1 A method for providing education to a student as to an internal anatomical structure, the method comprising: continually tracking over time a living subject with a motion tracking device external to the subject; continually estimating the subject's real-time skeletal pose and position over the time based on data obtained from the motion tracking device and continually determining the real-time pose and position of the subject's internal anatomical structure over the time based on the estimated skeletal pose and position; continually adjusting a three-dimensional model of the internal anatomical structure to match the determined pose and position; and continually projecting images of the internal anatomical structure onto the subject in registration with the actual internal anatomical structure of the subject to illustrate the location of the structure within the subject to a student, wherein the images of the internal anatomical structure remain in registration with the subject's actual internal anatomical structure as the subject moves because of the continual tracking, adjusting, and projecting. ### Claim 2 (depends on 1) The method of claim 1, wherein the subject is a human subject and the anatomical structure is an internal anatomical structure. ### Claim 3 (depends on 2) The method of claim 2, wherein the internal anatomical structure is an internal organ. ### Claim 4 (depends on 2) The method of claim 2, wherein the internal anatomical structure is a bone. ### Claim 5 (depends on 1) The method of claim 1, wherein continually projecting images comprises continually updating the images projected onto the subject so that the anatomical structure maintains its registration with the actual anatomical structure as the subject moves. ### Claim 6 (depends on 1) The method of claim 1, further comprising detecting a gesture made by the subject and performing in response an action associated with the gesture. ### Claim 7 (depends on 6) The method of claim 6, wherein the gesture comprises the subject pointing to a particular anatomical structure in the projected image. ### Claim 8 (depends on 6) The method of claim 6, wherein the action comprises projecting onto the subject a particular type of anatomical structure associated with the gesture. ### Claim 9 (depends on 6) The method of claim 6, further comprising quizzing the subject as to the anatomical structure. ### Claim 10 A non-transitory computer-readable medium that stores an image projection system configured to provide education to a student as to an internal anatomical structure, comprising: logic configured to continually estimate a skeletal pose and position of a living subject over time from data collected by continually tracking the subject with a motion tracking device external to the subject and to continually determine the real-time pose and position of the subject's internal anatomical structure over the time based on the estimated skeletal pose and position; logic configured to continually adjust a three-dimensional model of the internal anatomical structure of the subject to match the determined pose and position; and logic configured to continually generate images of the internal anatomical structure for projection onto the subject in registration with the actual internal anatomical structure of the subject to illustrate the location of the structure within the subject to a student, wherein the images of the internal anatomical structure remain in registration with the subject's actual internal anatomical structure as the subject moves because of the continual tracking, adjusting, and generating. ### Claim 11 (depends on 10) The non-transitory computer-readable medium of claim 10, wherein the logic configured to estimate a skeletal pose and position comprises logic configured to estimate the skeletal pose and position from images captured with a motion tracking device configured to track movement of the subject. ### Claim 12 (depends on 10) The non-transitory computer-readable medium of claim 10, wherein the logic configured to continually generate images is configured to continually update the images to be projected onto the subject so that the anatomical structure maintains its registration with the actual anatomical structure as the subject moves. ### Claim 13 (depends on 10) The non-transitory computer-readable medium of claim 10, further comprising logic configured to detect gestures made by the subject and to perform actions associated with the gestures. ### Claim 14 (depends on 10) The non-transitory computer-readable medium of claim 10, further comprising logic configured to quiz the subject about the anatomical structures. ### Claim 15 A system for providing education to a student as to an internal anatomical structure, the system comprising: an external motion tracking device configured to continually track a living subject over time; a projector configured to continually project images onto the subject; and a computing device configured to (i) continually receive data from the motion tracking device, continually estimate a skeletal pose and position of the subject from the data, and continually determine the real-time pose and position of the subject's internal anatomical structure over the time based on the estimated skeletal pose and position, (ii) continually adjust a three-dimensional model of the internal anatomical structure to match the determined pose and position, and (iii) continually provide updated images of the internal anatomical structure to the projector for projection onto the subject in registration with the actual internal anatomical structure of the subject to illustrate the location of the structure within the subject to a student, wherein the images of the internal anatomical structure remain in registration with the subject's actual internal anatomical structure as the subject moves because of the continual determining, adjusting, and providing. ### Claim 16 (depends on 15) The system of claim 15, wherein the motion tracking device is configured to capture images of the subject and the computing device is configured to estimate the skeletal pose and position from the captured images. ### Claim 17 (depends on 15) The system of claim 15, wherein the computing device is further configured to detect gestures made by the subject and to perform in response actions associated with the gestures. ### Claim 18 (depends on 15) The system of claim 15, wherein the computing device is further configured to quiz the subject as to the anatomical structures.