# US 2021/0279960 A1 · Photogrammetry of building using machine learning based inference > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0279960 A1 - Document ID: US20210279960A1 - Status: Published application - Kind: A1 - Application number: 17/191,731 - Filed: 2021-03-04 - Published: 2021-09-09 - Earliest claimed: 2020-03-05 - Assignees: Topcon Corporation - Applicants: Topcon Corporation - Inventors: Yasufumi FUKUMA; Satoshi YANOBE - Web page: https://patentgrep.com/patent/US20210279960A1 ## CPC - G06T 19/003: {Navigation within 3D models or images} - G06F 18/214: Generating training patterns; Bootstrap methods, e.g. bagging or boosting - G06T 2210/04: Architectural design, interior design - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract The data set receiving unit 13 of the information processing apparatus 1 of an aspect example receives a data set that includes at least BIM data. The route setting processor 151 sets a route, which is arranged inside and/or outside a virtual building represented by the BIM data, based on the data set received. The virtual image set generating processor 152 generates a virtual image set of the virtual building along the route, based on the received data set and the set route. The inference model creating processor 153 creates an inference model by applying machine learning with training data that includes at least the generated virtual image set to a neural network. The inference model created is used to identify data of a building material from data acquired by measuring a building. ## Claims ### Claim 1 An information processing apparatus comprising: a data set receiving circuit configured to receive a data set that includes at least building information modeling (BIM) data; a route setting processor configured to set a route arranged inside and/or outside a virtual building represented by the BIM data, based on the data set; a virtual image set generating processor configured to generate a virtual image set of the virtual building along the route, based on the data set and the route; and an inference model creating processor configured to create an inference model used to identify data of a building material from measured data of a building, by applying machine learning with training data that includes at least the virtual image set to a neural network. ### Claim 2 (depends on 1) The information processing apparatus of claim 1, wherein the data set further includes installation date information that shows an installation date of a virtual building material recorded in the BIM data, and measurement date information that shows a measurement date of a building, and the inference model creating processor creates an inference model by applying machine learning corresponding to the installation date and the measurement date to the neural network. ### Claim 3 (depends on 2) The information processing apparatus of claim 2, further comprising a BIM data creating processor configured to create first BIM data corresponding to a state in which only a virtual building material whose installation date is same as or earlier than the measurement date is installed, based on the data set, wherein the route setting processor sets a first route based on the first BIM data, the virtual image set generating processor generates a first virtual image set based on the first BIM data and the first route, and the inference model creating processor creates an inference model corresponding to the measurement date by applying machine learning with first training data that includes at least the first virtual image set to the neural network. ### Claim 4 (depends on 2) The information processing apparatus of claim 2, further comprising a BIM data creating processor configured to create second BIM data corresponding to a state in which only a virtual building material whose installation date is same as or earlier than the measurement date is installed, based on the data set, wherein the virtual image set generating processor generates a second virtual image set based on the second BIM data and the route, and the inference model creating processor creates an inference model corresponding to the measurement date by applying machine learning with second training data that includes at least the second virtual image set to the neural network. ### Claim 5 (depends on 1) The information processing apparatus of claim 1, wherein the route setting processor sets the route such that a distance from the route to a virtual building material recorded in the BIM data belongs to a predetermined allowable range. ### Claim 6 (depends on 1) The information processing apparatus of claim 1, wherein the route setting processor sets, as the route, any of a one dimensional area, a two dimensional area, and a three dimensional area. ### Claim 7 An information processing apparatus comprising: a data set receiving circuit configured to receive a data set that includes at least building information modeling (BIM) data and route information, the route information representing a route arranged inside and/or outside a virtual building represented by the BIM data; a virtual image set generating processor configured to generate a virtual image set of interior and/or exterior of the virtual building along the route, based on the data set; and an inference model creating processor configured to create an inference model used to identify data of a building material from measured data of a building, by applying machine learning with training data that includes at least the virtual image set to a neural network. ### Claim 8 An information processing apparatus comprising: a virtual image set receiving circuit configured to receive a virtual image set along a predetermined route, the virtual image set representing interior and/or exterior of a virtual building represented by building information modeling (BIM) data; and an inference model creating processor configured to create an inference model used to identify data of a building material from measured data of a building, by applying machine learning with training data that includes at least the virtual image set to a neural network. ### Claim 9 (depends on 1) The information processing apparatus of claim 1, wherein the data set further includes texture information that represents a state of a surface of a virtual building material, and the information processing apparatus further comprising a first information combining processor configured to generate first combined information by combining the texture information with the virtual image set, wherein the inference model creating processor applies machine learning with training data that includes at least the first combined information to the neural network. ### Claim 10 (depends on 1) The information processing apparatus of claim 1, wherein the data set further includes obstacle information, and the information processing apparatus further comprising a second information combining processor configured to generate second combined information by combining the obstacle information with the virtual image set, wherein the inference model creating processor applies machine learning with training data that includes at least the second combined information to the neural network. ### Claim 11 (depends on 1) The information processing apparatus of claim 1, further comprising a label assigning processor configured to assign a label relating to a virtual building material recorded in the BIM data to the virtual image set, wherein the inference model creating processor applies supervised learning with training data that includes at least the virtual image set and the label to the neural network. ### Claim 12 (depends on 1) The information processing apparatus of claim 1, wherein the neural network includes a convolutional neural network. ### Claim 13 (depends on 1) The information processing apparatus of claim 1, further comprising a transmitting circuit configured to transmit the inference model created by the inference model creating processor to a mobile object configured to perform measurement while autonomously moving inside and/or outside a building. ### Claim 14 An information processing apparatus comprising: a memory that stores an inference model created by applying, to a neural network, machine learning with training data that includes at least a virtual image set along a predetermined route arranged inside and/or outside a virtual building represented by building information modeling (BIM) data; and a processor configured to perform data processing using the inference model, wherein the processor includes: a feature extractor configured to extract a feature from measured data of a building; and an identifier configured to identify data of a building material based on the feature. ### Claim 15 A method of creating an inference model, comprising: preparing a data set that includes at least building information modelling (BIM) data; setting a route arranged inside and/or outside a virtual building represented by the BIM data, based on the data set; generating a virtual image set of the virtual building along the route, based on the data set and the route; and creating an inference model used to identify data of a building material from measured data of a building, by applying machine learning with training data that includes at least the virtual image set to a neural network. ### Claim 16 (depends on 15) The method of claim 15, wherein the data set further includes installation date information that shows an installation date of a virtual building material recorded in the BIM data, and measurement date information that shows a measurement date of a building, and the creating the inference model includes creating an inference model by applying machine learning corresponding to the installation date and the measurement date to the neural network. ### Claim 17 (depends on 16) The method of claim 16, further comprising creating first BIM data corresponding to a state in which only a virtual building material whose installation date is same as or earlier than the measurement date is installed, based on the data set, wherein the setting the route includes setting a first route based on the first BIM data, the generating the virtual image set includes generating a first virtual image set based on the first BIM data and the first route, and the creating the inference model includes creating an inference model corresponding to the measurement date by applying machine learning with first training data that includes at least the first virtual image set to the neural network. ### Claim 18 (depends on 16) The method of claim 16, further comprising creating second BIM data corresponding to a state in which only a virtual building material whose installation date is same as or earlier than the measurement date is installed, based on the data set, wherein the generating the virtual image set includes generating a second virtual image set based on the second BIM data and the route, and the creating the inference model includes creating an inference model corresponding to the measurement date by applying machine learning with second training data that includes at least the second virtual image set to the neural network. ### Claim 19 (depends on 15) The method of claim 15, wherein the setting the route includes setting a route such that a distance from the route to a virtual building material recorded in the BIM data belongs to a predetermined allowable range. ### Claim 20 (depends on 15) The method of claim 15, wherein the setting the route includes setting any of a one dimensional area, a two dimensional area, and a three dimensional area as a route. ### Claim 21 A method of creating an inference model, comprising: preparing a data set that includes at least building information modeling (BIM) data and route information, the route information representing a route arranged inside and/or outside a virtual building represented by the BIM data; generating a virtual image set of interior and/or exterior of the virtual building along the route, based on the data set; and creating an inference model used to identify data of a building material from measured data of a building, by applying machine learning with training data that includes at least the virtual image set to a neural network. ### Claim 22 A method of creating an inference model, comprising: preparing a virtual image set along a predetermined route, the virtual image set representing interior and/or exterior of a virtual building represented by building information modeling (BIM) data; and creating an inference model used to identify data of a building material from measured data of a building, by applying machine learning with training data that includes at least the virtual image set to a neural network. ### Claim 23 (depends on 15) The method of claim 15, wherein the data set further includes texture information that represents a state of a surface of a virtual building material, and the method further comprising generating first combined information by combining the texture information with the virtual image set, wherein the creating the inference model includes applying machine learning with training data that includes at least the first combined information to the neural network. ### Claim 24 (depends on 15) The method of claim 15, wherein the data set further includes obstacle information, and the method further comprising generating second combined information by combining the obstacle information with the virtual image set, wherein the creating the inference model includes applying machine learning with training data that includes at least the second combined information to the neural network. ### Claim 25 (depends on 15) The method of claim 15, further comprising assigning a label relating to a virtual building material recorded in the BIM data to the virtual image set, wherein the creating the inference model includes applying supervised learning with training data that includes at least the virtual image set and the label to the neural network. ### Claim 26 (depends on 15) The method of claim 15, wherein the neural network includes a convolutional neural network. ### Claim 27 (depends on 15) The method of claim 15, further comprising transmitting the inference model to a mobile object configured to perform measurement while autonomously moving inside and/or outside a building. ### Claim 28 A method of processing information, comprising: receiving an inference model created by applying, to a neural network, machine learning with training data that includes at least a virtual image set along a predetermined route arranged inside and/or outside a virtual building represented by building information modeling (BIM) data; extracting a feature from measured data of a building, by a processor configured to perform data processing using the inference model; and identifying data of a building material based on the feature, by the processor. ### Claim 29 (depends on 15) A computer-readable non-transitory recording medium that records a program causing a computer to execute the method of claim 15. ### Claim 30 (depends on 15) A computer-readable non-transitory recording medium that records a program causing a computer to execute a process of identifying data of a building material from measured data of a building using the inference model created by the method of claim 15.