Three dimensional virtual room-based user interface for a home automation system

A non-transitory electronic device readable medium having software encoded thereon, the software, when executed on one or more processors of one or more electronic devices, operable to:

• render and display a user-navigable three-dimensional (3-D) virtual room from a perspective defined by a virtual camera based on data from at least one of a plurality of two-dimensional (2-D) images of a physical room captured from different respective positions and orientations in the physical room, wherein the user-navigable 3-D virtual room includes depictions of one or more devices present in the physical room,
• receive an explicit navigation command or implicit action from a user;
• in response to the explicit navigation command or implicit action, translate or rotate the virtual camera to alter a position or an orientation of the virtual camera;
• re-render and display the user-navigable 3-D virtual room from a new perspective defined by the altered position or orientation of the virtual camera, wherein the new perspective does not coincide with the position or orientation in the physical room from which any of the 2-D images were captured, and the user-navigable 3-D virtual room is re-rendered by blending data from multiple 2-D images captured from different positions or orientations to show the user-navigable 3-D virtual room from the new perspective;
• receive a user interaction associated with the user-navigable 3-D virtual room;
• in response to the user interaction, change a state of a device in the physical room.

A method for controlling a home automation system using a user-navigable three-dimensional (3-D) virtual room that corresponds to a physical room, comprising:

• rendering and displaying, by a control application (app) executing on an electronic device, the user-navigable 3-D virtual room from a perspective defined by a virtual camera, wherein the control app renders the user-navigable 3-D virtual room based on data from a plurality of images of the physical room captured from different respective positions in the physical room;
• receiving an explicit navigation command or implicit action from a user;
• in response to the explicit navigation command or implicit action, translating or rotating the virtual camera, by the control app, to alter a position or an orientation of the virtual camera;
• re-rendering and displaying, by the control app, the user-navigable 3-D virtual room from a new perspective defined by the altered position or orientation, wherein the new perspective does not coincide with the position in the physical room from which any of the images were captured;
• receiving a user interaction associated with the user-navigable 3-D virtual room;
• and
• in response to the user interaction, causing the home automation system to change a state of a device in the physical room.

A non-transitory electronic device readable medium having software encoded thereon, the software, when executed on one or more processors of one or more electronic devices, operable to:

• generate a user-navigable 3-D virtual room by stitching a plurality of images of the physical room captured from different respective positions in the physical room and a 3-D space model of the physical room together to link image data to corresponding locations in the 3-D space model, tagging depictions of one or more devices visible in the stitched images and 3-D space model with hit regions, and assigning appearance changes to the one or more devices in the tagged, stitched images and 3-D space model;
• render and display the user-navigable 3-D virtual room from a perspective defined by a virtual camera;
• receive an explicit navigation command or implicit action from a user;
• in response to the explicit navigation command or implicit action, translate or rotate the virtual camera to alter a position or an orientation of the virtual camera;
• re-render and display the user-navigable 3-D virtual room from a new perspective defined by the altered position or orientation, wherein the new perspective does not coincide with the position in the physical room from which any of the images were captured;
• receive a user interaction associated with the user-navigable 3-D virtual room;
• and
• in response to the user interaction, cause the home automation system to change a state of a device in the physical room.