# US 12,118,682 B2 · Systems and methods for collecting, locating and visualizing sensor signals in extended reality > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,118,682 B2 - Document ID: US12118682B2 - Status: Granted - Kind: B2 - Application number: 18/241,278 - Filed: 2023-09-01 - Published: 2023-12-28 - Granted: 2024-10-15 - Earliest claimed: 2019-05-10 - Assignees: Badvr, Inc. - Applicants: Badvr, Inc. - Inventors: Jad Meouchy; Suzanne Ramona Borders - Web page: https://patentgrep.com/patent/US12118682B2 ## CPC - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G06T 7/33: using feature-based methods - G06T 19/20: Editing of three-dimensional [3D] images, e.g. changing shapes or colours, aligning objects or positioning parts - G06T 2200/04: involving 3D image data - G06T 2207/10028: Range image; Depth image; 3D point clouds - G06T 2207/20021: Dividing image into blocks, subimages or windows - G06T 2219/2004: Aligning objects, relative positioning of parts - H04W 24/04: Arrangements for maintaining operational condition - H04W 24/08: Testing, {supervising or monitoring} using real traffic - H04W 24/10: Scheduling measurement reports {; Arrangements for measurement reports} - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72) ## Abstract Systems and methods for rendering one or more different types of datasets is described. An exemplar method includes: (i) obtaining a first type of dataset, wherein each first data value within the first type of dataset is associated with one or more three-dimensional coordinates, which define a location or region in real space; (ii) obtaining a second type of dataset, wherein each second data value within the second type of dataset is associated with one or more of the three-dimensional coordinates; (iii) spatializing the first type of dataset to create a first type of spatialized dataset; (iv) spatializing the second type of dataset to create a second type of spatialized dataset; (v) aligning the first type of spatialized dataset with the second type of spatialized dataset to create an enhanced three-dimensional spatialized environment; and (vi) rendering the enhanced three-dimensional spatialized environment on a display component. ## Claims ### Claim 1 A method for rendering two different types of datasets, said method comprising: obtaining, by displacing one or more first sensors within a real space, a first type of dataset, wherein each first data value within said first type of dataset is associated with one or more three-dimensional coordinates, which define a location or region in real space; obtaining, by displacing one or more second sensors within said real space, a second type of dataset, wherein each second data value within said second type of dataset is associated with one or more of said three-dimensional coordinates, wherein type of data in said second type of dataset is different from said first type of dataset, and wherein said at least one of said first type of dataset or second type of dataset represents an intangible property of said real space; spatializing, using a plurality of said three-dimensional coordinates, said first type of dataset to create a first type of spatialized dataset; spatializing, using said plurality of said three-dimensional coordinates, said second type of dataset to create a second type of spatialized dataset; aligning said first type of spatialized dataset with said second type of spatialized dataset to create an enhanced three-dimensional spatialized environment; and rendering, using a rendering engine and on a display component, said enhanced three-dimensional spatialized environment. ### Claim 2 (depends on 1) The method for rendering two different types of datasets of claim 1, wherein said spatializing said second type of dataset is carried out using an attribute spatializing module residing on an eyewear or on an external processor disposed external to said eyewear, wherein said external processor being communicatively coupled to said eyewear, and said method further comprising: spatially partitioning, using said attribute spatializing module, said real space into plurality of discrete subdivisions, wherein said space being defined using said plurality of three-dimensional location coordinates; and integrating said discrete subdivisions to create a spatialized model of said real space, wherein said second data values being distributed, based upon said spatialized model, to create said second type of spatialized dataset. ### Claim 3 (depends on 1) The method for rendering two different types of datasets of claim 1, wherein said aligning said first type of spatialized data with said second type of spatialized data is carried out using an aligning module residing on an eyewear or on an external processor disposed external to said eyewear, and wherein said external processor being communicatively coupled to said eyewear. ### Claim 4 (depends on 1) The method for rendering two different types of datasets of claim 1, wherein said aligning includes using one or more common spatial features present in said first type of spatialized dataset and said second type of spatialized dataset to create said enhanced three-dimensional spatialized environment. ### Claim 5 (depends on 1) The method for rendering two different types of datasets of claim 1, wherein in said rendering, said rendering engine residing on an eyewear or on an external processor disposed external to said eyewear, and wherein said external processor being communicatively coupled to said eyewear. ### Claim 6 (depends on 1) The method for rendering two different types of datasets of claim 1, wherein said display component being integrated into a client device, which is at least one device chosen from a group comprising AR/VR headset, smartphone, and computer. ### Claim 7 (depends on 1) The method for rendering two different types of datasets of claim 1, wherein said obtaining said first type of dataset and said obtaining said second type of dataset is carried out by displacing multiple said first sensors and multiple said second sensors such that each of said first sensors and said second sensors are disposed on a AR/VR headset and said rendering includes rendering on said display component associated with said AR/VR headsets. ### Claim 8 (depends on 7) The method for rendering two different types of datasets of claim 7, wherein said displacing of multiple said first sensors and multiple said second sensors is carried out using multiple users, each wearing a different AR/VR headset. ### Claim 9 (depends on 8) The method for rendering two different types of datasets of claim 8, wherein said real space is subdivided into partitions and each of said users is displaced within a different subdivided portion. ### Claim 10 (depends on 8) The method for rendering two different types of datasets of claim 8, wherein said spatializing said first type of data set and/or said spatializing said second type of dataset is carried out by one or more of said AR/VR headset. ### Claim 11 (depends on 7) The method for rendering two different types of datasets of claim 7, wherein said different AR/VR headsets are communicatively coupled to a single network. ### Claim 12 (depends on 11) The method for rendering two different types of datasets of claim 11, wherein said obtaining said first type of dataset includes merging, from each of multiple said first sensors and using said single network, each first data value and said associated one or more three-dimensional coordinates and obtaining said second type of dataset includes merging, from each of multiple said sensors and using said single network, each second data value and said associated one or more three-dimensional coordinates. ### Claim 13 (depends on 1) The method for rendering two different types of datasets of claim 1, wherein said enhanced three-dimensional spatialized environment, rendering on said display, is a three-dimensional replication of said real space. ### Claim 14 A method for rendering an attribute value dataset and an image dataset, said method comprising: obtaining, using an optical sensor that is displaced within a real space and a ground positioning component, three-dimensional pixelated data values or three-dimensional voxelated data values for one or more corresponding three-dimensional location coordinates, wherein each of said corresponding three-dimensional location coordinates being associated with at least one associated three-dimensional pixelated data values or at least one associated three-dimensional voxelated data values; obtaining, using at least one non-optical sensor that is displaced within said real space, one or more different types of attribute values for one or more of said corresponding three-dimensional location coordinates, wherein each of said corresponding three-dimensional location coordinates being associated with attribute values of at least one type; spatializing, using a plurality of said corresponding three-dimensional location coordinates, said associated three-dimensional pixelated data values or said associated three-dimensional voxelated data values to create a spatialized image dataset; spatializing, using said plurality of said corresponding three-dimensional location coordinates, said associated attribute values of at least said type to create a spatialized attribute value dataset of at least said type; aligning said image spatialized dataset with said spatialized attribute value dataset to create an enhanced three-dimensional spatialized environment; and rendering, using a rendering engine and on a display component, said enhanced three-dimensional spatialized environment. ### Claim 15 (depends on 14) The method for rendering said attribute value dataset and said image dataset of claim 14, wherein said spatializing to create said spatialized image dataset and/or said spatializing to create said spatialized attribute value dataset includes using a microphone for collecting audio data present inside a real space, and operating said microphone in conjunction with a filter, to increase spatial resolution of said spatialized model for said space. ### Claim 16 (depends on 14) The method for rendering said attribute value dataset and said image dataset of claim 14, further comprising storing one or more different types of said associated attribute values and said corresponding three-dimensional coordinates into a database. ### Claim 17 (depends on 14) The method for rendering said attribute value dataset and said image dataset of claim 14, wherein using at least one said non-optical sensor includes using multiple non-optical sensors installed on different client devices that includes a smartphone or a wearable. ### Claim 18 (depends on 17) The method for rendering said attribute value dataset and said image dataset of claim 17, wherein said obtaining one or more different types of attribute values includes obtaining said attribute values of at least one type at said plurality of said corresponding three-dimensional location coordinates, such that each of said corresponding three-dimensional location coordinates being associated with attribute values of at least one said type. ### Claim 19 (depends on 14) The method for rendering said attribute value dataset and said image dataset of claim 14, wherein said aligning being implemented, using a processor, based upon one or more common vertices present in spatialized image dataset and in spatialized attribute value dataset, and located inside a real space being defined by said plurality of said corresponding three-dimensional location coordinates. ### Claim 20 (depends on 14) The method for rendering said attribute value dataset and said image dataset of claim 14, wherein said rendering includes: representing said associated attribute values of a particular type obtained at a location inside a real space with virtual objects; comparing said associated attribute values of said particular type to a threshold attribute value of said particular type; assigning said virtual objects a first color, if said associated attribute values of said particular type is greater than said threshold attribute value of said particular type; assigning said virtual objects a second color, if said associated attribute values of said particular type is less than said threshold attribute value of said particular type, and wherein said second color is different from said first color; and rendering said virtual objects at objects rendering locations, in said enhanced three-dimensional spatialized environment, that correspond to locations of said corresponding three-dimensional location coordinates that are associated with said associated attribute values of said particular type. ### Claim 21 (depends on 20) The method for rendering said attribute value dataset and said image dataset of claim 20, wherein said spatializing to create said spatialized attribute value dataset includes interpolating, using two or more of said associated attribute values of said particular type and two or more of said corresponding three-dimensional location coordinates, to compute an intermediate associated attribute value of said particular type, and wherein a corresponding intermediate three-dimensional location coordinate, disposed between two or more of said corresponding three-dimensional location coordinates, being associated with said intermediate associated attribute value of a particular type. ### Claim 22 (depends on 21) The method for rendering said attribute value dataset and said image dataset of claim 21, further comprising rendering an intermediate virtual object at an intermediate object location, in said enhanced three-dimensional spatialized environment, that corresponds to a location of said corresponding intermediate three-dimensional location coordinate associated with said intermediate associated attribute value of said particular type. ### Claim 23 (depends on 20) The method for rendering said attribute value dataset and said image dataset of claim 20, wherein said spatializing to create said spatialized attribute value dataset includes extrapolating, using two or more of said associated attribute values of a particular type and two or more of said corresponding three-dimensional location coordinates, to compute a predicted associated attribute value of said particular type, wherein a corresponding exterior three-dimensional location coordinate, disposed outside of said real space and positioned on a linear trajectory that intersects two or more of said corresponding three-dimensional location coordinates, being associated with said predicted associated attribute value of said particular type, and wherein said real space being defined using said plurality of said corresponding three-dimensional location coordinates. ### Claim 24 (depends on 23) The method for rendering said attribute value dataset and said image dataset of claim 23, in said rendering, further comprising rendering exterior virtual objects at exterior object locations, in said enhanced three-dimensional spatialized environment, corresponding to locations of said corresponding exterior three-dimensional location coordinate associated with said predicted associated attribute values of said particular type. ### Claim 25 (depends on 14) The method for rendering said attribute value dataset and said image dataset of claim 14, wherein said spatializing to create a spatialized attribute value dataset of at least said type includes creating an electronic representation of a wire mesh extending a real space defined using said plurality of said corresponding three-dimensional location coordinates, and said method further comprising using said electronic representation of said wire mesh to compute an unknown attribute value of at least said type at a defined location in said real space.