# US 12,216,438 B2 · Systems and methods for pairing smart devices based on user interactions > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,216,438 B2 - Document ID: US12216438B2 - Status: Granted - Kind: B2 - Application number: 17/775,051 - Filed: 2020-11-06 - Published: 2022-12-08 - Granted: 2025-02-04 - Earliest claimed: 2019-11-06 - Assignees: Hubbell Incorporated - Applicants: Hubbell Incorporated - Inventors: Eric Ferguson; David Edward Davis - Web page: https://patentgrep.com/patent/US12216438B2 ## CPC - G05B 15/02: electric - G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object - G05B 2219/2642: Domotique, domestic, home control, automation, smart house - G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G) - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Systems and methods for pairing smart devices are provided. A method can include receiving image data indicative of an environment. The method can include identifying a plurality of devices physically located in the environment and generating a virtual representation of the environment based on the image data. The virtual representation can include a device representation for each respective device of the plurality of devices physically located in the environment. The method can include providing for display, via a display device, data indicative of a user interface. The user interface can present the virtual representation of the environment. The method can include receiving user input indicative of a pairing intention of a first device and second device and, in response, initiating a pairing operation between the first and second device. In this manner, the systems and methods can allow user to easily pair devices using a simple user-interaction. ## Claims ### Claim 1 A computer-implemented method for pairing devices comprising: receiving, by one or more computing devices, image data indicative of an environment from a camera configured to capture the image data; identifying, by the one or more computing devices, a plurality of devices physically located in the environment; receiving, by the one or more computing devices, location data via a location-based resource, wherein the location data includes user location data; generating, by the one or more computing devices, a virtual representation of the environment based, at least in part, on the image data, wherein the virtual representation comprises a device representation for each respective device of the plurality of devices physically located in the environment; providing for display, by the one or more computing devices via a display device, data indicative of a user interface, the user interface presenting the virtual representation of the environment; receiving, by the one or more computing devices, user input indicative of a pairing intention of a first device of the plurality of devices and a second device of the plurality of devices, wherein the user input includes a drag and drop motion across the user interface; initiating, by the one or more computing devices, a pairing operation between the first device and the second device based, at least in part, on the user input indicative of the pairing intention of the first device of the plurality of devices and the second device of the plurality of devices, wherein the pairing operation includes providing a pairing request indicative of the pairing operation between the first device and the second device to a remote server communicatively connected to at least one of the first device and the second device; receiving, by the one or more computing devices, confirmation data indicative of a completed pairing operation between the first device and the second device and displaying an indication of the confirmation data within the user interface wherein the confirmation data is indicative of a communicative connection between the first device and the second device, and receiving, by the one or more computing devices, user input associated with a termination operation and initiate the termination operation between the first device and the second device, wherein the termination operation initiates a termination of the communicative connection between the first device and the second device, wherein each respective device of the plurality of devices is associated with device data indicative of one or more device features and a device location, wherein identifying the plurality of devices physically located in the environment includes obtaining the device data associated with each respective device of the plurality of devices, and identifying the plurality of devices physically located in the environment based, at least in part, on the user location data and the device data associated with each respective device of the plurality of devices, and wherein the plurality of devices comprise at least one or more of a switch, a luminary, a ceiling fan, a media device, a audio device, a cleaning device, or a smart hub. ### Claim 2 (depends on 1) The computer-implemented method of claim 1, wherein the device data indicative of one or more device features includes device functionalities. ### Claim 3 (depends on 2) The computer-implemented method of claim 2, wherein providing for display, by the one or more computing devices via the display device, data indicative of the user interface comprises: generating, by the one or more computing devices, the device representation for each respective device of the plurality of devices based, at least in part, on the device data associated with each respective device; and providing for display, by the one or more computing devices via the display device, the data indicative of the user interface, the user interface presenting the device representation for each respective device of the plurality of devices within the virtual representation of the environment. ### Claim 4 (depends on 3) The computer-implemented method of claim 3, wherein the device representation for each respective device is provided for display at a virtual position within the virtual representation of the environment based, at least in part, on the location data associated with the respective device. ### Claim 5 (depends on 1) The computer-implemented method of claim 1, wherein the drag and drop motion includes a continuous touch input beginning at the first device and ending at the second device. ### Claim 6 (depends on 1) The computer-implemented method of claim 1, wherein initiating, by the one or more computing devices, the pairing operation between the first device and the second device comprises: obtaining, by the one or more computing devices, at least one of first communication data associated with the first device or second communication data associated with the second device; and providing, by the one or more computing devices, at least one of first pairing instructions indicative of the second communication data to the first device or second pairing instructions indicative of the first communication data to the second device. ### Claim 7 (depends on 1) The computer-implemented method of claim 1, wherein the remote server obtains communication data associated with the first device and the second device. ### Claim 8 (depends on 1) The computer-implemented method of claim 1, the confirmation data indicative of a completed pairing operation is provided by the remote server. ### Claim 9 (depends on 1) The computer-implemented method of claim 1, wherein the confirmation data indicative of a completed pairing operation is provided by at least one of the first device or the second device. ### Claim 10 (depends on 9) The computer-implemented method of claim 9, the user input associated with a termination operation includes a swiping motion. ### Claim 11 A user device comprising: one or more processors; a camera configured to capture image data; and a location-based resource configured to receive user location data associated with the user device; a display device; and one or more tangible, non-transitory, computer readable media that collectively store instructions that when executed by the one or more processors cause the user device to perform an operation configured to, receive image data indicative of an environment, identify a plurality of devices physically located in the environment, generate a virtual representation of the environment based, at least in part, on the image data, provide for display, via the display device, data indicative of a user interface, the user interface presenting the virtual representation of the environment, wherein the virtual representation comprises a device representation for each respective device of the plurality of devices physically located in the environment, receive user input indicative of a pairing intention of a first device of the plurality of devices and a second device of the plurality of devices, wherein the user input comprises a drag and drop motion across the user interface, initiate a pairing operation between the first device and the second device based, at least in part, on the user input indicative of the pairing intention of the first device of the plurality of devices and the second device of the plurality of devices, wherein the pairing operation includes providing a pairing request indicative of the pairing operation between the first device and the second device to a remote server communicatively connected to at least one of the first device and the second device, wherein the confirmation data is indicative of a communicative connection between the first device and the second device, receive confirmation data indicative of a completed pairing operation between the first device and the second device and displaying an indication of the confirmation data within the user interface, and receive user input associated with a termination operation and initiate the termination operation between the first device and the second device, wherein the termination operation initiates a termination of the communicative connection between the first device and the second device, wherein each respective device of the plurality of devices is associated with device data indicative of one or more device features and a device location, wherein identifying the plurality of devices physically located in the environment includes obtaining the device data associated with each respective device of the plurality of devices, and identifying the plurality of devices physically located in the environment based, at least in part, on the user location data and the device data associated with each respective device of the plurality of devices, and wherein the plurality of devices comprise at least one or more of a switch, a luminary, a ceiling fan, a media device, a audio device, a cleaning device, or a smart hub. ### Claim 12 (depends on 11) The user device of claim 11, wherein the location-based resource includes a global positioning sensor. ### Claim 13 (depends on 11) The user device of claim 11 further comprising a communication interface configured to communicate with one or more remote servers, wherein the device data for each respective device of the plurality of devices is stored on the one more remote servers. ### Claim 14 (depends on 3) The user device of claim 3, wherein the device data for each respective device of the plurality of devices is previously received via user input at the user device. ### Claim 15 (depends on 11) The user device of claim 11, wherein the operation includes receiving, via the camera, device image data indicative of a respective device of the plurality of devices, and determining the device data for the respective device based, at least in part, on the device image data for the respective device of the plurality of devices. ### Claim 16 A system for pairing smart devices comprising: one or more processors; a camera configured to capture image data; and a location-based resource configured to provide location data; a display device; and one or more tangible, non-transitory, computer readable media that collectively store instructions that when executed by the one or more processors cause the user device to perform an operation configured to, receive image data indicative of an environment, identify a plurality of devices physically located in the environment, generate a virtual representation of the environment based, at least in part, on the image data, provide for display, via the display device, data indicative of a user interface, the user interface presenting the virtual representation of the environment, wherein the virtual representation comprises a device representation for each respective device of the plurality of devices physically located in the environment, receive user input indicative of a pairing intention of a first device of the plurality of devices and a second device of the plurality of devices, wherein the user input comprises a drag and drop motion across the user interface, initiate a pairing operation between the first device and the second device based, at least in part, on the user input indicative of the pairing intention of the first device of the plurality of devices and the second device of the plurality of devices, wherein the pairing operation includes providing a pairing request indicative of the pairing operation between the first device and the second device to a remote server communicatively connected to at least one of the first device and the second device, receive confirmation data indicative of a completed pairing operation between the first device and the second device and displaying an indication of the confirmation data within the user interface wherein the confirmation data is indicative of a communicative connection between the first device and the second device, and receive user input associated with a termination operation and initiate the termination operation between the first device and the second device, wherein the termination operation initiates a termination of the communicative connection between the first device and the second device, wherein each respective device of the plurality of devices is associated with device data indicative of one or more device features and a device location, wherein identifying the plurality of devices physically located in the environment includes obtaining the device data associated with each respective device of the plurality of devices, and identifying the plurality of devices physically located in the environment based, at least in part, on the location data and the device data associated with each respective device of the plurality of devices, and wherein the plurality of devices comprise at least one or more of a switch, a luminary, a ceiling fan, a media device, a audio device, a cleaning device, or a smart hub. ### Claim 17 (depends on 16) The system of claim 16, wherein the camera and the display device are part of a hand-held user device. ### Claim 18 (depends on 17) The system of claim 17, wherein the device data for each respective device of the plurality of devices is previously received via user input at the user device. ### Claim 19 (depends on 16) The system of claim 16, wherein the user input associated with a termination operation includes a swiping motion. ### Claim 20 (depends on 16) The system of claim 16, wherein the operation includes receiving, via the camera, device image data indicative of a respective device of the plurality of devices, and determining the device data for the respective device based, at least in part, on the device image data for the respective device of the plurality of devices.