# US 2019/0246071 A1 · Building Monitoring System

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2019/0246071 A1
- Document ID: US20190246071A1
- Status: Published application
- Kind: A1
- Application number: 15/891,338
- Filed: 2018-02-07
- Published: 2019-08-08
- Earliest claimed: 2018-02-07
- Assignees: Osram Sylvania Inc.
- Inventors: Nancy H. Chen; Barry Stout
- Web page: https://patentgrep.com/patent/US20190246071A1

## CPC

- G10L 25/51: for comparison or discrimination
- G06K 7/10722: {Photodetector array or CCD scanning}
- G06K 7/1417: {2D bar codes}
- G06V 20/20: in augmented reality scenes
- G06V 20/52: Surveillance or monitoring of activities, e.g. for recognising suspicious objects (recognising microscopic objects G06V20/69)
- H04N 7/181: {for receiving images from a plurality of remote sources}
- H04N 7/185: {from a mobile camera, e.g. for remote control}
- H04N 23/661: Transmitting camera control signals through networks, e.g. control via the Internet
- H04N 23/90: Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
- G10L: SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G06K: GRAPHICAL DATA READING (image or video recognition or understanding G06V); PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
- G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION

## Abstract

A building monitoring system includes a plurality of image capture devices deployed to obtain images of target building systems, the target building systems being associated with optical landmarks visible by the image capture devices. A plurality of audio beacons are configured to output audio landmarks. A plurality of audio capture devices are deployed to obtain sound clips of the target building systems, each sound clip of the target building systems including at least one of the audio landmarks from the audio beacons. A data analysis system is configured to receive the images of the target building systems and identify each target building system from at least one of the optical landmarks and the audio landmarks.

## Claims

### Claim 1

A building monitoring system, comprising:
a plurality of image capture devices deployed to obtain images of target building systems, the target building systems being associated with optical landmarks visible by the image capture devices;
a plurality of audio beacons configured to output audio landmarks;
a plurality of audio capture devices deployed to obtain sound clips of the target building systems, each sound clip of the target building systems including at least one of the audio landmarks from the plurality of audio beacons; and
a data analysis system configured to:
receive the images of the target building systems;
receive the sound clips of the target building systems; and
identify each of the target building systems using at least one of the optical landmarks and the audio landmarks.

### Claim 2 (depends on 1)

The building monitoring system of claim 1, wherein the plurality of image capture devices are fixed relative to the target building systems.

### Claim 3 (depends on 1)

The building monitoring system of claim 1, wherein the optical landmarks are bar codes or Quick Response (QR) codes.

### Claim 4 (depends on 1)

The building monitoring system of claim 1, wherein the audio landmarks are audio signatures in a frequency range above approximately 20 kHz to be outside a human audible frequency range.

### Claim 5 (depends on 1)

The building monitoring system of claim 1, further comprising a mobile audio and video capture device configured to obtain images of the target building systems and optical landmarks, and configured to obtain sound clips of the target building systems and the audio beacons.

### Claim 6 (depends on 5)

The building monitoring system of claim 5, wherein the data analysis system is further configured to receive the images and sound clips from the mobile audio and video capture device and sort the images from the mobile audio and video capture device using the optical landmarks, and to sort the sound clips from the mobile audio and video capture device using the audio landmarks.

### Claim 7 (depends on 1)

The building monitoring system of claim 1, further comprising a mobile device executing a building monitoring system application configured to specify a new target building system to be monitored by the building monitoring system.

### Claim 8 (depends on 7)

The building monitoring system of claim 7, wherein the building monitoring system application is configured to specify the new target building system by:
acquiring a test image of the new target building system to be monitored;
determining whether image properties of the test image are satisfactory; and
determining whether one of the optical landmarks is visible in the test image.

### Claim 9 (depends on 7)

The building monitoring system of claim 7, wherein the building monitoring system application is configured to specify the new target building system by:
receiving user input comprising characterizing information about the new target building system to be monitored.

### Claim 10 (depends on 7)

The building monitoring system of claim 7, wherein the building monitoring system application is configured to specify the new target building system by:
acquiring a test sound clip of the new target building system to be monitored;
determining whether audio properties of the test sound clip are satisfactory; and
determining whether one of the audio landmarks is audible in the test sound clip.

### Claim 11 (depends on 1)

The building monitoring system of claim 1, wherein the data analysis system is further configured to:
analyze the images for an anomaly in the operation of the target building systems; and
in response to detecting an anomaly in the operation of a first target building system, instruct a first image capture device in the plurality of image capture devices to obtain additional images of the first target building system.

### Claim 12 (depends on 1)

The building monitoring system of claim 1, wherein the data analysis system is further configured to:
analyze the sound clips for an anomaly in the operation of the target building systems; and
in response to detecting an anomaly in the operation of a first target building system, instruct a first audio capture device in the plurality of audio capture devices to obtain additional sound clips of the first target building system.

### Claim 13 (depends on 1)

The building monitoring system of claim 1, wherein the data analysis system is further configured to:
analyze the images for an anomaly in the operation of the target building systems; and
in response to detecting an anomaly in the operation of a first target building system, instruct a first audio capture device in the plurality of audio capture devices to obtain additional sound clips of the first target building system.

### Claim 14 (depends on 1)

The building monitoring system of claim 1, wherein the data analysis system is further configured to:
analyze the sound clips for an anomaly in the operation of the target building systems; and
in response to detecting an anomaly in the operation of a first target building system, instruct a first image capture device in the plurality of image capture devices to obtain additional images of the first target building system.

### Claim 15 (depends on 1)

The building monitoring system of claim 1, wherein the data analysis system is further configured to:
analyze the images for an anomaly in the operation of the target building systems; and
in response to detecting an anomaly in the operation of a first target building system, instruct a mobile audio and video capture device to obtain a sound clip of the first target building system or to obtain additional images of the first target building system.

### Claim 16 (depends on 1)

The building monitoring system of claim 1, wherein the data analysis system is further configured to:
analyze the sound clips for an anomaly in the operation of the target building systems; and
in response to detecting an anomaly in the operation of a first target building system, instruct a mobile audio and video capture device to obtain a sound clip of the first target building system or to obtain additional images of the first target building system.

### Claim 17 (depends on 1)

The building monitoring system of claim 1, wherein:
at least one of the image capture devices and at least one of the audio capture devices are configured to cooperatively collect a video clip; and
the data analysis system is further configured to receive the video clip and sort the video clip by detecting one of the optical landmarks or one of the audio landmarks in the video clip.

### Claim 18 (depends on 17)

The building monitoring system of claim 17, wherein:
when the video clip includes one of the optical landmarks, the data analysis system is further configured to extract images and sound from the video clip and sort the images and sound using the optical landmark; and
when the video clip includes one of the audio landmarks, the data analysis system is configured to extract images and sound from the video clip and sort the images and sound using the audio landmark.

### Claim 19

A method of building monitoring, comprising:
receiving an image of a target building system from an image capture device, the image including a picture of the target building system and an optical landmark associated with the target building system;
identifying the target building system from the optical landmark;
extracting information about a first monitored aspect of the target building system from the image of the target building system; and
comparing the extracted information about the first monitored aspect of the target building system with previously extracted information about the first monitored aspect of the target building system from previously received images of the target building system.

### Claim 20 (depends on 19)

The method of building monitoring of claim 19, further comprising:
receiving a sound clip of the target building system from an audio capture device, the sound clip including sound produced by the target building system and an audio landmark from an audio beacon proximate the target building system;
extracting information about a second monitored aspect of the target building system from the sound clip of the target building system; and
comparing the extracted information about the second monitored aspect of the target building system with previously extracted information about the second monitored aspect of the target building system from previously received sound clips of the target building system.
