# US 12,192,027 B2 · Camera-based commissioning and control of devices in a load control system

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 12,192,027 B2
- Document ID: US12192027B2
- Status: Granted
- Kind: B2
- Application number: 18/107,267
- Filed: 2023-02-08
- Published: 2023-10-19
- Granted: 2025-01-07
- Earliest claimed: 2018-08-02
- Assignees: Lutron Technology Company LLC
- Applicants: Lutron Technology Company LLC
- Inventors: Johnathan Jessen; Duheng Liang
- Web page: https://patentgrep.com/patent/US12192027B2

## CPC

- H04L 12/2809: {indicating that an appliance service is present in a home automation network (monitoring functionality H04L43/0817; discovery or management thereof, e.g. service location protocol [SLP] or web services, H04L67/51)}
- G05D 1/00: Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots (drive control systems specially adapted for autonomous road vehicles B60W60/00)
- G05D 1/0219: {ensuring the processing of the whole working surface}
- G05D 1/0253: {extracting relative motion information from a plurality of images taken successively, e.g. visual odometry, optical flow (determining position or orientation from images G06T7/70)}
- G05D 1/027: {comprising intertial navigation means, e.g. azimuth detector (inertial navigation G01C21/16; inertial navigation combined with non-inertial navigation instruments G01C21/165)}
- G05D 1/0274: {using mapping information stored in a memory device (navigation using map-matching G01C21/30)}
- G05D 1/227: Handing over between remote control and on-board control; Handing over between remote control arrangements
- G05D 1/2437: {Extracting relative motion information}
- G05D 1/245: using dead reckoning
- G05D 1/246: using environment maps, e.g. simultaneous localisation and mapping [SLAM]
- G05D 1/648: Performing a task within a working area or space, e.g. cleaning
- G06T 7/70: Determining position or orientation of objects or cameras (camera calibration G06T7/80)
- G06T 11/00: Two-dimensional [2D] image generation
- H05B 47/105: in response to determined parameters
- H05B 47/19: via wireless transmission
- H05B 47/195: the transmission using visible or infrared light
- H05B 47/1965: {using handheld communication devices}
- H05B 47/198: {Grouping of control procedures or address assignation to light sources}
- H05B 47/199: {Commissioning of light sources}
- H04L 2012/2841: {Wireless}
- Y02B 20/40: Control techniques providing energy savings, e.g. smart controller or presence detection
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- G05D: SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- H05B: ELECTRIC HEATING; ELECTRIC LIGHT SOURCES NOT OTHERWISE PROVIDED FOR; CIRCUIT ARRANGEMENTS FOR ELECTRIC LIGHT SOURCES, IN GENERAL
- Y02B: CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO BUILDINGS, e.g. HOUSING, HOUSE APPLIANCES OR RELATED END-USER APPLICATIONS

## Abstract

Lighting control systems may be commissioned for programming and/or control with the aid of an autonomous mobile device. Design software may be used to create a floor plan of how the lighting control system may be designed. The design software may generate floor plan identifiers for each lighting fixture, or group of lighting fixtures. During commissioning of the lighting control system, the autonomous mobile device may be used to help identify the lighting devices that have been installed in the physical space. The autonomous mobile device may receive a communication from each lighting control device that indicates a unique identifier of the lighting control device. The unique identifier may be communicated by visible light communication (VLC) or RF communication. The unique identifier may be associated with the floor plan identifier for communication of digital messages to lighting fixtures installed in the locations indicated in the floor plan identifier.

## Claims

### Claim 1

A method of commissioning a lighting control system via an autonomous mobile device, the method comprising:
determining an area in which the autonomous mobile device is located;
capturing an image using a camera of the autonomous mobile device, wherein the image includes a plurality of lighting fixtures in the area;
determining, based on the image, a location and an orientation of the autonomous mobile device relative to the plurality of lighting fixtures;
determining an order for commissioning the plurality of lighting fixtures in the image; and
automatically commissioning the plurality of lighting fixtures based on the determined order, wherein automatically commissioning the plurality of lighting fixtures comprises automatically associating respective unique identifiers of a plurality of lighting control devices with respective floor plan identifiers of the lighting fixtures in accordance with the determined order, wherein each of the plurality of lighting control devices is associated with a respective one of the plurality of lighting fixtures, wherein each unique identifier is configured to identify a respective lighting control device in messages transmitted to control a lighting load in the respective lighting fixture, and wherein each floor plan identifier identifies a respective location of a lighting fixture on a floor plan of the area.

### Claim 2 (depends on 1)

The method of claim 1, wherein automatically commissioning the plurality of lighting fixtures based on the determined order further comprises:
determining a respective floor plan identifier for each of the plurality of lighting fixtures based on the determined order; and
receiving, in accordance with the determined order, the respective unique identifiers of the plurality of lighting control devices.

### Claim 3 (depends on 2)

The method of claim 2, wherein receiving the respective unique identifiers further comprises receiving the unique identifiers of the lighting control devices via visible light communication (VLC) signals.

### Claim 4 (depends on 2)

The method of claim 2, wherein receiving the respective unique identifiers further comprises receiving the unique identifiers of the lighting control devices via RF signals.

### Claim 5 (depends on 1)

The method of claim 1, further comprising, prior to capturing the image, moving the autonomous mobile device to the area without the use of input from a user.

### Claim 6 (depends on 1)

The method of claim 1, wherein determining the area in which the autonomous mobile device is located further comprises automatically determining the area using a real-time locating system.

### Claim 7 (depends on 1)

The method of claim 1, wherein determining the orientation of the autonomous mobile device further comprises automatically determining the orientation by identifying predefined objects within the image.

### Claim 8 (depends on 7)

The method of claim 7, wherein the predefined objects within the image comprise at least one of the corners of a room, lighting fixtures, or windows.

### Claim 9 (depends on 1)

The method of claim 1, wherein determining the orientation of the autonomous mobile device further comprises determining the orientation based on at least two predefined objects within an image.

### Claim 10 (depends on 1)

The method of claim 1, wherein determining the orientation of the autonomous mobile device further comprises determining the orientation based on information from at least one of an accelerometer or a gyroscope of the autonomous mobile device.

### Claim 11 (depends on 1)

The method of claim 1, wherein determining the floor plan data comprises mapping a position of one or more objects within the area relative to a predetermined point.

### Claim 12 (depends on 1)

The method of claim 1, wherein determining the orientation of the autonomous mobile device further comprises determining the orientation by measuring a distance between two or more predefined objects within an image of the location and identifying the distance as corresponding to a distance in a floor plan of the location.

### Claim 13 (depends on 12)

The method of claim 12, further comprising using the measured distance to identify a lighting fixture of the plurality of lighting fixtures as corresponding to a floor plan lighting fixture in the floor plan.

### Claim 14 (depends on 13)

The method of claim 13, wherein associating the unique identifiers of the lighting control devices with the respective floor plan identifiers comprises storing associations between the unique identifiers and the respective floor plan identifiers in a memory of the autonomous mobile device.

### Claim 15 (depends on 1)

The method of claim 1, further comprising sending a digital message that comprises control instructions configured to control a first lighting control device of the plurality of lighting control devices, and wherein the digital message comprises at least one of the unique identifier of the first lighting control device or the floor plan identifier associated with the first lighting control device.

### Claim 16

A method of commissioning a lighting control system via an autonomous mobile device, the method comprising:
receiving, from a remote device, a command to move the autonomous mobile device to a position within an area;
capturing an image using a camera of the autonomous mobile device, wherein the image comprises a plurality of lighting fixtures within the area;
determining an order for commissioning the plurality of lighting fixtures in the image; and
commissioning the plurality of lighting fixtures based on the determined order, wherein commissioning the plurality of lighting fixtures comprises associating unique identifiers of a plurality of lighting control devices with respective floor plan identifiers of the lighting fixtures in accordance with the determined order, wherein each of the plurality of lighting control devices is associated with a respective one of the plurality of lighting fixtures, wherein each unique identifier is configured to identify a respective lighting control device in messages transmitted to control an amount of power provided to respective lighting loads in the plurality of lighting fixtures, and wherein each floor plan identifiers identifies a respective position of one of the plurality of lighting fixtures relative to other objects on a floor plan of the area.

### Claim 17 (depends on 16)

The method of claim 16, wherein commissioning the plurality of lighting fixtures based on the determined order further comprises receiving, in accordance with the determined order, the respective unique identifiers of the plurality of lighting control devices.

### Claim 18 (depends on 17)

The method of claim 17, wherein commissioning the plurality of lighting fixtures based on the determined order further comprises:
determining, based on the image, respective locations of the plurality of lighting fixtures with respect to the autonomous mobile device; and
determining the respective floor plan identifiers based on the position of the autonomous mobile device and the respective locations of the plurality of lighting fixtures with respect to the autonomous mobile device.

### Claim 19 (depends on 18)

The method of claim 18, wherein determining the respective floor plan identifiers comprises selecting one or more floor plan identifiers from a predefined list.

### Claim 20 (depends on 18)

The method of claim 18, wherein determining the respective floor plan identifiers comprises sending a request to the remote device to prompt a user to provide the floor plan identifiers.

### Claim 21 (depends on 17)

The method of claim 17, wherein receiving the respective unique identifiers further comprises receiving the unique identifiers of the lighting control devices via visible light communication (VLC) signals.

### Claim 22 (depends on 17)

The method of claim 17, wherein receiving the respective unique identifiers further comprises receiving the unique identifiers of the lighting control devices via RF signals.

### Claim 23 (depends on 16)

The method of claim 16, further comprising sending a digital message comprising control instructions configured to control a first lighting control device of the plurality of lighting control devices, and wherein the digital message comprises at least one of the unique identifier of the first lighting control device or the floor plan identifier associated with the first lighting control device.

### Claim 24

An autonomous mobile device comprising:
a camera; and
a control circuit configured to:
determine an area in which the autonomous mobile device is located;
capture an image using the camera, wherein the image includes a plurality of lighting fixtures in the area;
determine, based on the image, a location and an orientation of the autonomous mobile device relative to the plurality of lighting fixtures;
determine an order for commissioning the plurality of lighting fixtures in the image; and
automatically commission the plurality of lighting fixtures based on the determined order, wherein the control circuit being configured to automatically commission the plurality of lighting fixtures comprises the control circuit being configured to automatically associate respective unique identifiers of a plurality of lighting control devices with respective floor plan identifiers of the lighting fixtures in accordance with the determined order, wherein each of the plurality of lighting control devices is associated with a respective one of the plurality of lighting fixtures, wherein each unique identifier is configured to identify a respective lighting control device in messages transmitted to control a lighting load in the respective lighting fixture, and wherein each floor plan identifier identifies a respective location of a lighting fixture on a floor plan of the area.

### Claim 25 (depends on 24)

The autonomous mobile device of claim 24, wherein the control circuit being configured to automatically commission the plurality of lighting fixtures based on the determined order comprises the control circuit being configured to:
determine a respective floor plan identifier for each of the plurality of lighting fixtures based on the determined order; and
receive, in accordance with the determined order, the respective unique identifiers of the plurality of lighting control devices.

### Claim 26 (depends on 24)

The autonomous mobile device of claim 24, wherein the control circuit being configured to determine the area in which the autonomous mobile device is located comprises the control circuit being configured to automatically determine the area using a real-time locating system.

### Claim 27 (depends on 24)

The autonomous mobile device of claim 24, wherein the control circuit being configured to determine the orientation of the autonomous mobile device comprises the control circuit being configured to automatically determine the orientation by identifying predefined objects within the image, wherein the predefined objects within the image comprise at least one of the corners of a room, lighting fixtures, or windows.

### Claim 28 (depends on 24)

The autonomous mobile device of claim 24, wherein the control circuit being configured to determine the orientation of the autonomous mobile device comprises the control circuit being configured to determine the orientation by measuring a distance between two or more predefined objects within an image of the location and identifying the distance as corresponding to a distance in a floor plan of the location.

### Claim 29 (depends on 24)

The autonomous mobile device of claim 24, wherein the control circuit is further configured to send a digital message that comprises control instructions configured to control a first lighting control device of the plurality of lighting control devices, wherein the digital message comprises at least one of the unique identifier of the first lighting control device or the floor plan identifier associated with the first lighting control device.

### Claim 30

Non-transitory computer readable media having instructions stored thereon that, when executed by a control circuit, cause the control circuit to:
determine an area in which an autonomous mobile device is located;
capturing an image using a camera, wherein the image includes a plurality of lighting fixtures in the area;
determine, based on the image, a location and an orientation of the autonomous mobile device relative to the plurality of lighting fixtures;
determine an order for commissioning the plurality of lighting fixtures in the image; and
automatically commission the plurality of lighting fixtures based on the determined order, wherein the instructions causing the control circuit to automatically commission the plurality of lighting fixtures comprises the instructions causing the control circuit to automatically associate respective unique identifiers of a plurality of lighting control devices with respective floor plan identifiers of the lighting fixtures in accordance with the determined order, wherein each of the plurality of lighting control devices is associated with a respective one of the plurality of lighting fixtures, wherein each unique identifier is configured to identify a respective lighting control device in messages transmitted to control a lighting load in the respective lighting fixture, and wherein each floor plan identifier identifies a respective location of a lighting fixture on a floor plan of the area.

### Claim 31 (depends on 30)

The non-transitory computer readable media of claim 30, wherein the instructions causing the control circuit to automatically commission the plurality of lighting fixtures based on the determined order comprises the instructions causing the control circuit to:
determine a respective floor plan identifier for each of the plurality of lighting fixtures based on the determined order; and
receive, in accordance with the determined order, the respective unique identifiers of the plurality of lighting control devices.

### Claim 32 (depends on 30)

The non-transitory computer readable media of claim 30, wherein the instructions causing the control circuit to determine the area in which the autonomous mobile device is located comprises the instructions causing the control circuit to automatically determine the area using a real-time locating system.

### Claim 33 (depends on 30)

The non-transitory computer readable media of claim 30, wherein the instructions causing the control circuit to determine the orientation of the autonomous mobile device comprises the instructions causing the control circuit to automatically determine the orientation by identifying predefined objects within the image, wherein the predefined objects within the image comprise at least one of the corners of a room, lighting fixtures, or windows.

### Claim 34 (depends on 30)

The non-transitory computer readable media of claim 30, wherein the instructions causing the control circuit to determine the orientation of the autonomous mobile device comprises the instructions causing the control circuit to determine the orientation by measuring a distance between two or more predefined objects within an image of the location and identifying the distance as corresponding to a distance in a floor plan of the location.

### Claim 35 (depends on 30)

The non-transitory computer readable media of claim 30, wherein the instructions further cause the control circuit to send a digital message that comprises control instructions configured to control a first lighting control device of the plurality of lighting control devices, wherein the digital message comprises at least one of the unique identifier of the first lighting control device or the floor plan identifier associated with the first lighting control device.

### Claim 36

An autonomous mobile device comprising:
a camera; and
a control circuit configured to:
receive, from a remote device, a command to move the autonomous mobile device to a position within an area;
capture an image using the camera, wherein the image comprises a plurality of lighting fixtures within the area;
determine an order for commissioning the plurality of lighting fixtures in the image; and
commission the plurality of lighting fixtures based on the determined order, wherein commissioning the plurality of lighting fixtures comprises associating unique identifiers of a plurality of lighting control devices with respective floor plan identifiers of the lighting fixtures in accordance with the determined order, wherein each of the plurality of lighting control devices is associated with a respective one of the plurality of lighting fixtures, wherein each unique identifier is configured to identify a respective lighting control device in messages transmitted to control an amount of power provided to respective lighting loads in the plurality of lighting fixtures, and wherein each floor plan identifiers identifies a respective position of one of the plurality of lighting fixtures relative to other objects on a floor plan of the area.

### Claim 37 (depends on 36)

The autonomous mobile device of claim 36, wherein the control circuit being configured to commission the plurality of lighting fixtures based on the determined order comprises the control circuit being configured to receive, in accordance with the determined order, the respective unique identifiers of the plurality of lighting control devices.

### Claim 38 (depends on 37)

The autonomous mobile device of claim 37, wherein the control circuit being configured to commission the plurality of lighting fixtures based on the determined order further comprises the control circuit being configured to:
determine, based on the image, respective locations of the plurality of lighting fixtures with respect to the autonomous mobile device; and
determine the respective floor plan identifiers based on the position of the autonomous mobile device and the respective locations of the plurality of lighting fixtures with respect to the autonomous mobile device.

### Claim 39

Non-transitory computer readable media having instructions stored thereon that, when executed by a control circuit, cause the control circuit to:
receive, from a remote device, a command to move an autonomous mobile device to a position within an area;
capture an image using a camera, wherein the image comprises a plurality of lighting fixtures within the area;
determine an order for commissioning the plurality of lighting fixtures in the image; and
commission the plurality of lighting fixtures based on the determined order, wherein commissioning the plurality of lighting fixtures comprises associating unique identifiers of a plurality of lighting control devices with respective floor plan identifiers of the lighting fixtures in accordance with the determined order, wherein each of the plurality of lighting control devices is associated with a respective one of the plurality of lighting fixtures, wherein each unique identifier is configured to identify a respective lighting control device in messages transmitted to control an amount of power provided to respective lighting loads in the plurality of lighting fixtures, and wherein each floor plan identifiers identifies a respective position of one of the plurality of lighting fixtures relative to other objects on a floor plan of the area.

### Claim 40 (depends on 39)

The non-transitory computer readable media of claim 39, wherein the instructions further cause the control circuit to send a digital message comprising control instructions configured to control a first lighting control device of the plurality of lighting control devices, and wherein the digital message comprises at least one of the unique identifier of the first lighting control device or the floor plan identifier associated with the first lighting control device.