# US 10,872,466 B1 · Creating improved 3D models from 2D data > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,872,466 B1 - Document ID: US10872466B1 - Status: Granted - Kind: B1 - Application number: 16/439,579 - Filed: 2019-06-12 - Published: 2020-12-17 - Granted: 2020-12-22 - Earliest claimed: 2019-06-12 - Assignees: Trimble Inc. - Applicants: Trimble Inc. - Inventors: Darrin Hurd - Web page: https://patentgrep.com/patent/US10872466B1 ## CPC - G06T 7/73: using feature-based methods - G06T 17/05: Geographic models - G06T 7/521: from laser ranging, e.g. using interferometry; from the projection of structured light - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06T 2207/30184: Infrastructure - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract Techniques for displaying underground assets using a portable electronic device. A camera image of a ground having a ground surface may be captured. A placement of a virtual pit in the camera image may be determined. A pit view comprising the virtual pit and a virtual representation of an underground asset, and a superimposed image comprising the camera image and the pit view may be generated. A top opening of the virtual pit may align with the ground surface in the camera image. A portion of the virtual representation of the underground asset may be shown inside the virtual pit and below the top opening of the virtual pit such that a distance between the top opening and the portion of the virtual representation of the underground asset inside the virtual pit may provide a visual indication of a depth of the underground asset relative to the ground surface. ## Claims ### Claim 1 A method for displaying underground assets using an electronic device, the method comprising: capturing a camera image of a ground having a ground surface; determining a placement of a virtual pit in the camera image; generating a pit view comprising the virtual pit and a virtual representation of an underground asset based on asset data associated with the underground asset, the virtual pit comprising at least one depth marker along a wall of the virtual pit; and generating a superimposed image comprising the camera image and the pit view such that: a top opening of the virtual pit aligns with the ground surface in the camera image; and a portion of the virtual representation of the underground asset is shown inside the virtual pit and below the top opening of the virtual pit such that a distance between the top opening and the portion of the virtual representation of the underground asset inside the virtual pit provides a visual indication of a depth of the underground asset relative to the ground surface. ### Claim 2 (depends on 1) The method of claim 1, wherein determining the placement of the virtual pit comprises: determining a ground plane; determining a location of an intersection of a virtual ray and the ground plane, wherein the virtual ray is projected from a center of a camera component configured to capture the camera image such that the intersection of the virtual ray and the ground plane is near a center of the camera image; and generating the virtual pit based on the location of the intersection. ### Claim 3 (depends on 2) The method of claim 2, wherein: determining the ground plane comprises determining, based on a geospatial position of the electronic device, an estimated ground plane at a predetermined distance below the electronic device; and determining the location of the intersection of the virtual ray and the ground plane comprises determining a location of an intersection of the virtual ray and the estimated ground plane. ### Claim 4 (depends on 2) The method of claim 2, wherein: the asset data comprises attributes of one or more underground assets; determining the ground plane comprises determining one or more collision planes, wherein each of the one or more collision planes is determined based on the attributes of the one or more underground assets such that each of the one or more collision planes overlays the ground surface and extends along an extension of a corresponding underground asset; and determining the location of the intersection of the virtual ray and the ground plane comprises determining a location of an intersection of the virtual ray and at least one of the one or more collision planes. ### Claim 5 (depends on 2) The method of claim 2, wherein: determining the ground plane comprises recognizing the ground plane based on the captured camera image. ### Claim 6 (depends on 1) The method of claim 1, wherein determining the placement of the virtual pit comprises: accessing a height map of the ground; aligning the height map with the ground surface of the camera image; determining a location of an intersection of a virtual ray and the height map, wherein the virtual ray is projected from a center of a camera component configured to capture the camera image such that the intersection of the virtual ray and the ground plane is near a center of the camera image; and generating the virtual pit based on the location of the intersection. ### Claim 7 (depends on 1) The method of claim 1, wherein determining the placement of the virtual pit comprises: determining a position of a point of interest in the camera image based on a distance to the point of interest measured by an electronic distance meter (EDM) device of the electronic device; and generating the virtual pit at the point of interest. ### Claim 8 (depends on 1) The method of claim 1, wherein determining the placement of the virtual pit comprises: generating a height map of the ground using an electronic distance meter (EDM) device of the electronic device; determining a location of an intersection of a virtual ray and the height map, wherein the virtual ray is projected from a center of a camera component configured to capture the camera image such that the intersection of the virtual ray and the ground plane is near a center of the camera image; and generating the virtual pit based on the location of the intersection. ### Claim 9 (depends on 1) The method of claim 1, wherein generating the pit view comprises: generating a first representation of the underground asset based at least in part on one of depth information or position information of the asset data for displaying inside the virtual pit; and generating a second representation of the underground asset based at least in part on one of the depth information or position information of the asset data for displaying at the ground surface in the superimposed image. ### Claim 10 (depends on 9) The method of claim 9, wherein generating the pit view further comprises: determining pixels of the first representation of the underground asset that are within a predetermined radius of a center of the virtual pit for displaying in the superimposed image; determining pixels of the second representation of the underground asset that are outside the predetermined radius of the center of the virtual pit for displaying in the superimposed image; and generating a pit mask configured to render any virtual representation inside the virtual pit and behind the pit mask transparent. ### Claim 11 An electronic device for displaying underground assets, the electronic device comprising: a global navigation satellite system (GNSS) receiver configured to determine a geospatial position of the GNSS receiver; and an electronic distance measuring (EDM) device configured to capture a distance to a point of interest, wherein the GNSS receiver and the EDM device are configured to be communicatively coupled to a camera via one or more processors, wherein the one or more processes are configured to perform operations comprising: capturing a camera image, using the camera, of a ground having a ground surface; determining a placement of a virtual pit in the camera image using at least one of the GNSS receiver and the EDM device; generating a pit view comprising the virtual pit and a virtual representation of an underground asset based on asset data associated with the underground asset, the virtual pit comprising at least one depth marker along a wall of the virtual pit; and generating a superimposed image comprising the camera image and the pit view such that: a top opening of the virtual pit aligns with the ground surface in the camera image; and a portion of the virtual representation of the underground asset is shown inside the virtual pit and below the top opening of the virtual pit such that a distance between the top opening and the portion of the virtual representation of the underground asset inside the virtual pit provides a visual indication of a depth of the underground asset relative to the ground surface. ### Claim 12 (depends on 11) The electronic device of claim 11, wherein determining the placement of the virtual pit comprises: determining a ground plane; determining a location of an intersection of a virtual ray and the ground plane, wherein the virtual ray is projected from a center of the camera such that the intersection of the virtual ray and the ground plane is near a center of the camera image; and generating the virtual pit based on the location of the intersection. ### Claim 13 (depends on 12) The electronic device of claim 12, wherein: determining the ground plane comprises determining, based on the geospatial position of the GNSS receiver, an estimated ground plane at a predetermined distance below the GNSS receiver; and determining the location of the intersection of the virtual ray and the ground plane comprises determining a location of an intersection of the virtual ray and the estimated ground plane. ### Claim 14 (depends on 12) The electronic device of claim 12, wherein: the asset data comprises attributes of one or more underground assets; determining the ground plane comprises determining one or more collision planes, wherein each of the one or more collision planes is determined based on attributes of the one or more underground assets such that each of the one or more collision planes overlays the ground surface and extends along an extension of a corresponding underground asset; and determining the location of the intersection of the virtual ray and the ground plane comprises determining a location of an intersection of the virtual ray and at least one of the one or more collision planes. ### Claim 15 (depends on 12) The electronic device of claim 12, wherein: determining the ground plane comprises recognizing the ground plane based on the captured camera image. ### Claim 16 (depends on 11) The electronic device of claim 11, wherein determining the placement of the virtual pit comprises: accessing a height map of the ground; aligning the height map with the ground surface of the camera image; determining a location of an intersection of a virtual ray and the height map, wherein the virtual ray is projected from a center of the camera such that the intersection of the virtual ray and the ground plane is near a center of the camera image; and generating the virtual pit based on the location of the intersection. ### Claim 17 (depends on 11) The electronic device of claim 11, wherein determining the placement of the virtual pit comprises: determining a position of a point of interest in the camera image based on a distance to the point of interest measured by the EDM device; and generating the virtual pit at the point of interest. ### Claim 18 (depends on 11) The electronic device of claim 11, wherein determining the placement of the virtual pit comprises: generating a height map of the ground using the EDM device; determining a location of an intersection of a virtual ray and the height map, wherein the virtual ray is projected from a center of the camera such that the intersection of the virtual ray and the ground plane is near a center of the camera image; and generating the virtual pit based on the location of the intersection. ### Claim 19 (depends on 11) The electronic device of claim 11, wherein generating the pit view comprises: generating a first representation of the underground asset based at least in part on one of depth information or position information of the asset data for displaying inside the virtual pit; and generating a second representation of the underground asset based at least in part on one of the depth information or position information of the asset data for displaying at the ground surface in the superimposed image. ### Claim 20 (depends on 19) The electronic device of claim 19, wherein generating the pit view further comprises: determining pixels of the first representation of the underground asset that are within a predetermined radius of a center of the virtual pit for displaying in the superimposed image; determining pixels of the second representation of the underground asset that are outside the predetermined radius of the center of the virtual pit for displaying in the superimposed image; and generating a pit mask configured to render any virtual representation inside the virtual pit and behind the pit mask transparent.