# US 9,239,892 B2 · X-ray vision for buildings > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 9,239,892 B2 - Document ID: US9239892B2 - Status: Granted - Kind: B2 - Application number: 14/146,176 - Filed: 2014-01-02 - Published: 2015-07-02 - Granted: 2016-01-19 - Earliest claimed: 2014-01-02 - Assignees: DPR Construction - Applicants: Justin Schmidt; Timothy Conroy - Inventors: Justin Schmidt; Timothy Conroy - Web page: https://patentgrep.com/patent/US9239892B2 ## CPC - G06F 30/13: Architectural design, e.g. computer-aided architectural design [CAAD] related to design of buildings, bridges, landscapes, production plants or roads - G06F 3/00: Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements - G06T 19/00: Manipulating three-dimensional [3D] models or images for computer graphics - G06T 2210/04: Architectural design, interior design - G06T 2219/004: Annotating, labelling - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Embodiments of the present invention include a method for generating a skeletal view of a building including receiving location information and spatial orientation information for a computing device. Further, a field of view may be determined based on the location information and spatial orientation information. The computing device may then receive information about a building at a location near the computing device, and identify a structure in the field of view of the computing device. Based on the information about the building, the structure, and the field of view, a graphical representation of the structure may be prepared and displayed. ## Claims ### Claim 1 A method for generating a skeletal view of an existing building in view of a computing device having a camera, the skeletal view being a representation of hidden structural components and visible structural components of the existing building, the method comprising: receiving, via a processor, location information associated with the computing device, the location information identifying a physical location of the computing device; receiving spatial orientation information for the computing device, the spatial orientation information identifying a physical orientation of the computing device; determining a field of view for the computing device using the location information and the spatial orientation information, the field of view including a three-dimensional (3D) location of the computing device and a direction of view from the camera of the computing device; receiving three-dimensional (3D) model information representing construction details of the existing building, the existing building being at a location proximate to the computing device, and the 3D model information including the hidden structural components and the visible structural components; comparing the field of view with the 3D model information; identifying a hidden structural component based on the comparison between the field of view and the 3D model information, the hidden structural component being hidden within at least one of a wall, a ceiling or a floor of the existing building; preparing a composite representation including a visible structural component and the hidden structural component; and displaying the composite representation on the computing device. ### Claim 2 (depends on 1) The method of claim 1 further comprising receiving an indication from a user identifying the hidden structural component included in the composite representation and displaying additional information about the hidden structural component, including one or more of the following: component dimensions, a part number for the component, and a manufacturer of the component. ### Claim 3 (depends on 1) The method of claim 1 wherein the location information associated with the computing device comprises GPS coordinates or a scanned code affixed to the existing building. ### Claim 4 (depends on 1) The method of claim 1 wherein the location information associated with the computing device comprises image data relating to a visual marker captured by the camera of the computing device. ### Claim 5 (depends on 1) The method of claim 1 wherein the location information associated with the computing device comprises information about signals from WiFi access points received at the computing device. ### Claim 6 (depends on 1) The method of claim 1 wherein the spatial orientation information comprises information from a gyroscope or a compass. ### Claim 7 (depends on 1) The method of claim 1 wherein the computing device comprises a telephone. ### Claim 8 (depends on 1) The method of claim 1 wherein the computing device comprises a tablet computer. ### Claim 9 (depends on 1) The method of claim 1 further comprising, receiving a user selection of a type of hidden structural component, and preparing the composite representation to include the selected type of hidden structural component and exclude other non-selected types of hidden structural components. ### Claim 10 (depends on 8) The method of claim 8 wherein the three-dimensional model comprises information collected by laser-scanning. ### Claim 11 (depends on 10) The method of claim 10 wherein the information collected by laser-scanning is combined with a to be constructed model to form an as constructed model. ### Claim 12 (depends on 1) The method of claim 1 wherein the visible structural component comprises a wall. ### Claim 13 (depends on 1) The method of claim 1 wherein the hidden structural component comprises one or more of the following: electrical wiring, ductwork, plumbing, studs, MEP components and structural support hidden within at least one of a wall, a ceiling or a floor of the existing building. ### Claim 14 A non-transitory computer-readable storage medium comprising a plurality of computer-readable instructions tangibly embodied on the computer-readable storage medium, which when executed by a data processor, provide a skeletal view of an existing building in view of a computing device having a camera, the skeletal view being a representation of hidden structural components and visible structural components of the existing building, the plurality of instructions comprising: instructions that cause the data processor to receive location information associated with the computing device, the location information identifying a physical location of the computing device; instructions that cause the data processor to receive spatial orientation information for the computing device, the spatial orientation information identifying a physical orientation of the computing device; instructions that cause the data processor to determine a field of view for the computing device, based on the location information and spatial orientation information, the field of view including a three-dimensional (3D) location of the computing device and a direction of view from the camera of the computing device; instructions that cause the data processor to receive three-dimensional (3D) model information representing construction details of the existing building, the existing building being at a location proximate to the computing device, and the 3D model information including the hidden structural components and the visible structural components; instructions that cause the data processor to compare the field of view with the 3D model information; instructions that cause the data processor to identify a hidden structural component based on the comparison between the field of view and the 3D model information, the hidden structural component being hidden within at least one of a wall, a ceiling or a floor of the existing building; instructions that cause the data processor to, generate a composite representation including visible structural components of the existing building and the hidden structural component; and instructions that cause the data processor to display the composite representation on the computing device. ### Claim 15 (depends on 14) The non-transitory computer-readable storage medium of claim 14 further comprising instructions that cause the data processor to receive an indication from a user identifying a hidden structural component included in the composite representation and displaying additional information about the hidden structural component, including one or more of the following: component dimensions, a part number for the component, and a manufacturer of the component. ### Claim 16 (depends on 14) The non-transitory computer-readable storage medium of claim 14 wherein the location information associated with the computing device comprises information about signals from WiFi access points received at the computing device. ### Claim 17 (depends on 14) The non-transitory computer-readable storage medium of claim 14 wherein the computing device comprises a tablet computer. ### Claim 18 (depends on 14) The non-transitory computer-readable storage medium of claim 14 wherein the composite representation includes image data of the visible structural component captured by the camera, and computer-generated graphics representing the hidden structural component. ### Claim 19 A system for generating a skeletal view of an existing building in view of a computing device having a camera, the skeletal view being a representation of hidden structural components and visible structural components of the existing building, the system comprising: a processor; and a memory disposed in communication with the processor and storing processor executable instructions, the instructions comprising instructions for: receiving location information associated with the computing device, the location information identifying a physical location of the computing device; receiving spatial orientation information for the computing device, the spatial orientation information identifying a physical orientation of the computing device; determining a field of view for the computing device using the location information and the spatial orientation information, the field of view including a three-dimensional (3D) location of the computing device and a direction of view from the camera of the computing device; receiving three-dimensional (3D) model information representing construction details of the existing building, the existing building being at a location proximate to the computing device; comparing the field of view with the 3D model information; identifying a hidden structural component based on the comparison between the field of view and the 3D model information, the hidden structural component being hidden within at least one of a wall, a ceiling or a floor of the existing building; preparing a composite representation including a visible structural component and the hidden structural component; and displaying the composite representation on a display of the system. ### Claim 20 (depends on 19) The system of claim 19 wherein the instructions further comprise instructions for receiving an indication from a user identifying the hidden structural component included in the composite representation and displaying additional information about the hidden structural component, including one or more of the following: component dimensions, a part number for the component, and a manufacturer of the component. ### Claim 21 (depends on 19) The system of claim 19 wherein the location information associated with the computing device comprises information about signals from WiFi access points received at the computing device. ### Claim 22 (depends on 19) The system of claim 19 wherein the computing device comprises a tablet computer. ### Claim 23 (depends on 19) The system of claim 19 wherein the 3D model information comprises a three-dimensional model of at least a part of the existing building. ### Claim 24 (depends on 1) The method of claim 1, wherein comparing the field of view with the 3D model information comprises aligning the 3D model with the 3D location and the direction of view to identify the hidden structural component.