# US 11,308,346 B2 · Sensor information integration system, sensor information integration method, and program > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 11,308,346 B2 - Document ID: US11308346B2 - Status: Granted - Kind: B2 - Application number: 16/964,886 - Filed: 2019-02-01 - Published: 2021-02-04 - Granted: 2022-04-19 - Earliest claimed: 2018-02-02 - Assignees: NEC Corporation - Applicants: NEC Corporation - Inventors: Toshiki Takeuchi - Web page: https://patentgrep.com/patent/US11308346B2 ## CPC - G06V 20/58: Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads - G01C 21/26: specially adapted for navigation in a road network - G01S 13/86: Combinations of radar systems with non-radar systems, e.g. sonar, direction finder - G01S 15/87: Combinations of sonar systems - G01S 17/87: Combinations of systems using electromagnetic waves other than radio waves - G06F 18/22: Matching criteria, e.g. proximity measures - G06F 18/2415: based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate - G06F 18/25: Fusion techniques - G06F 18/251: {of input or preprocessed data} - G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80) - G06V 10/803: {of input or preprocessed data} - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G01C: MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY (measuring liquid level G01F; radio navigation, determining distance or velocity by use of propagation effects, e.g. Doppler effects, propagation time, of radio waves, analogous arrangements using other waves G01S) - G01S: RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract A system includes a first analysis apparatus, a second analysis apparatus, and a sensor fusion apparatus. The individual analysis apparatuses analyze outputs of their respective sensors and generate preciseness information about preciseness of the positions of their respective objects. The sensor fusion apparatus calculates the distance between a determination target object and an association candidate object per directional axis that is determined from a position of a sensor corresponding to the determination target object. The sensor fusion apparatus normalizes the distance calculated per directional axis by using the preciseness information corresponding to the determination target object. The sensor fusion apparatus calculates the distance between the determination target object and the association candidate object by using the distance normalized per directional axis as a post-normalization distance. The sensor fusion apparatus determines whether the association candidate object matches the determination target object by using the post-normalization distance. ## Claims ### Claim 1 A sensor information integration system, comprising: a first analysis apparatus that analyzes an output(s) of a first sensor(s), estimates a position of a first object, and generates first preciseness information about preciseness of the estimated position of the first object; a second analysis apparatus that analyzes an output(s) of a second sensor(s), estimates a position of a second object, and generates second preciseness information about preciseness of the estimated position of the second object; and a sensor fusion apparatus that integrates output results of the first and second analysis apparatuses; wherein, the sensor fusion apparatus includes: a distance calculation part that determines, based on the first and second preciseness information, one of the first and second objects to be a determination target object and the other object to be an association candidate object, calculates a distance between the determination target object and the association candidate object per directional axis that is determined from a position of a sensor corresponding to the determination target object, normalizes the distance calculated per directional axis by using the preciseness information corresponding to the determination target object, and calculates a distance between the determination target object and the association candidate object as a post-normalization distance by using the distance normalized per directional axis; and an association determination part that determines whether the association candidate object matches the determination target object by using the post-normalization distance. ### Claim 2 (depends on 1) The sensor information integration system according to claim 1; wherein at least one of the first and second analysis apparatuses outputs a positional estimation result and preciseness information about a corresponding object(s). ### Claim 3 (depends on 1) The sensor information integration system according to claim 1; wherein the distance calculation part sets a straight line that connects a sensor and an object as a first directional axis and sets a straight line that is perpendicular to the first directional axis and that passes through the object as a second directional axis; and wherein the distance calculation part calculates a distance between the determination target object and the association candidate object for an individual one of the first directional axis and the second directional axis. ### Claim 4 (depends on 1) The sensor information integration system according to claim 1; wherein the distance calculation part sets an object corresponding to the first or second preciseness information that represents higher reliability to be the determination target object. ### Claim 5 (depends on 1) The sensor information integration system according to claim 1; wherein an individual one of the first and second preciseness information includes at least a standard deviation(s); and wherein the sensor fusion apparatus further includes a threshold calculation part which calculates a threshold(s) that the association determination part uses for determining whether the association candidate object matches the determination target object by using the standard deviation(s) included in the first and second preciseness information. ### Claim 6 (depends on 5) The sensor information integration system according to claim 5; wherein the threshold calculation part calculates an absolute threshold by using the first or second preciseness information that represents lower reliability. ### Claim 7 (depends on 6) The sensor information integration system according to claim 6; wherein, based on a comparison performed between the post-normalization distance and the absolute threshold, the association determination part determines whether the association candidate object matches the determination target object. ### Claim 8 (depends on 5) The sensor information integration system according to claim 5; wherein, when the association candidate object exists in plurality, the threshold calculation part calculates a relative threshold by using preciseness information corresponding to the respective association candidate objects. ### Claim 9 A sensor information integration method of a sensor fusion apparatus, comprising: wherein the sensor fusion apparatus is connected to a first analysis apparatus that analyzes an output(s) of a first sensor(s), estimates a position of a first object, and generates first preciseness information about preciseness of the estimated position of the first object and a second analysis apparatus that analyzes an output(s) of a second sensor(s), estimates a position of a second object, and generates second preciseness information about preciseness of the estimated position of the second object and that integrates output results of the first and second analysis apparatuses, determining, based on the first and second preciseness information, one of the first and second objects to be a determination target object and the other object to be an association candidate object; calculating a distance between the determination target object and the association candidate object per directional axis that is determined from a position of a sensor corresponding to the determination target object; normalizing the distance calculated per directional axis by using the preciseness information corresponding to the determination target object; calculating a distance between the determination target object and the association candidate object as a post-normalization distance by using the distance normalized per directional axis; and causing the sensor fusion apparatus to determine whether the association candidate object matches the determination target object by using the post-normalization distance. ### Claim 10 (depends on 9) The sensor information integration method according to claim 9; wherein at least one of the first and second analysis apparatuses outputs a positional estimation result and preciseness information about a corresponding object(s). ### Claim 11 (depends on 9) The sensor information integration method according to claim 9, comprising: setting a straight line that connects a sensor and an object as a first directional axis; setting a straight line that is perpendicular to the first directional axis and that passes through the object as a second directional axis; and calculating a distance between the determination target object and the association candidate object for an individual one of the first directional axis and the second directional axis. ### Claim 12 (depends on 9) The sensor information integration method according to claim 9, comprising: setting an object corresponding to the first or second preciseness information that represents higher reliability to be the determination target object. ### Claim 13 (depends on 9) The sensor information integration method according to claim 9, comprising: calculating a threshold(s) for determining whether the association candidate object matches the determination target object by using standard deviation(s) included in the first and second preciseness information. ### Claim 14 (depends on 13) The sensor information integration method according to claim 13, comprising: calculating an absolute threshold by using the first or second preciseness information that represents lower reliability. ### Claim 15 A non-transient computer readable medium storing a program, causing a computer mounted on a sensor fusion apparatus that is connected to a first analysis apparatus that analyzes an output(s) of a first sensor(s), estimates a position of a first object, and generates first preciseness information about preciseness of the estimated position of the first object and a second analysis apparatus that analyzes an output(s) of a second sensor(s), estimates a position of a second object, and generates second preciseness information about preciseness of the estimated position of the second object and integrates output results of the first and second analysis apparatuses, to perform processing for: determining, based on the first and second preciseness information, one of the first and second objects to be a determination target object and the other object to be an association candidate object; calculating a distance between the determination target object and the association candidate object per directional axis that is determined from a position of a sensor corresponding to the determination target object; normalizing the distance calculated per directional axis by using the preciseness information corresponding to the determination target object; calculating a distance between the determination target object and the association candidate object as a post-normalization distance by using the distance normalized per directional axis; and determining whether the association candidate object matches the determination target object by using the post-normalization distance. ### Claim 16 (depends on 15) The non-transient computer readable medium storing the program according to claim 15; wherein at least one of the first and second analysis apparatuses outputs a positional estimation result and preciseness information about a corresponding object(s). ### Claim 17 (depends on 15) The non-transient computer readable medium storing the program according to claim 15, the program comprising: setting a straight line that connects a sensor and an object as a first directional axis; setting a straight line that is perpendicular to the first directional axis and that passes through the object as a second directional axis; and calculating a distance between the determination target object and the association candidate object for an individual one of the first directional axis and the second directional axis. ### Claim 18 (depends on 15) The non-transient computer readable medium storing the program according to claim 15, the program comprising: setting an object corresponding to the first or second preciseness information that represents higher reliability to be the determination target object. ### Claim 19 (depends on 15) The non-transient computer readable medium storing the program according to claim 15, the program comprising: calculating a threshold(s) for determining whether the association candidate object matches the determination target object by using standard deviation(s) included in the first and second preciseness information. ### Claim 20 (depends on 19) The non-transient computer readable medium storing the program according to claim 19, the program comprising: calculating an absolute threshold by using the first or second preciseness information that represents lower reliability.