# US 2019/0320146 A1 · Integrated accessory control user interface > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2019/0320146 A1 - Document ID: US20190320146A1 - Status: Published application - Kind: A1 - Application number: 16/455,559 - Filed: 2019-06-27 - Published: 2019-10-17 - Earliest claimed: 2016-06-12 - Assignees: Apple Inc. - Applicants: Apple Inc. - Inventors: Cyrus D. Irani; Kevin P. McLaughlin; Anush G. Nadathur; Arun G. Mathias; Vinay A. Ramaswamy; Christopher D. Soli; Patrick L. Coffman; Stephen O. Lemay - Web page: https://patentgrep.com/patent/US20190320146A1 ## CPC - H04N 7/186: {Video door telephones} - G06F 3/048: Interaction techniques based on graphical user interfaces [GUI] - G07C 9/00309: {operated with bidirectional data transmission between data carrier and locks} - G08B 21/182: {Level alarms, e.g. alarms responsive to variables exceeding a threshold} - H04L 12/2803: {Home automation networks} - H04L 12/2818: {from a device located outside both the home and the home network (access arrangements H04L12/2856; for remote control or remote monitoring of applications H04L67/025; arrangements for transmitting signals characterised by the use of a wireless eletrical link G08C17/00; telephonic communication systems adapted for combination with remote control systems H04M11/007)} - H04L 12/282: {based on user interaction within the home (receiver circuitry for displaying additional information being controlled by a remote control apparatus H04N21/42204)} - H04L 43/0864: Round trip delays - H04L 47/283: in response to processing delays, e.g. caused by jitter or round trip time [RTT] - H04L 65/1069: Session establishment or de-establishment - H04L 65/61: for supporting one-way streaming services, e.g. Internet radio - H04L 67/125: involving control of end-device applications over a network - H04L 67/141: Setup of application sessions (admission control or resource allocation in data switching networks H04L47/70) - H04L 67/146: Markers for unambiguous identification of a particular session, e.g. session cookie or URL-encoding - H04N 5/77: between a recording apparatus and a television camera - H04N 7/18: Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast - H04N 23/661: Transmitting camera control signals through networks, e.g. control via the Internet - H04W 76/11: Allocation or use of connection identifiers - G05B 19/042: using digital processors (G05B19/05 takes precedence) - G05B 2219/2642: Domotique, domestic, home control, automation, smart house - G08B 25/008: {Alarm setting and unsetting, i.e. arming or disarming of the security system} - H04N 7/183: {for receiving images from a single remote source} - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G07C: TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE - G08B: SIGNALLING SYSTEMS, e.g. PERSONAL CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS - H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M) - H04W: WIRELESS COMMUNICATION NETWORKS (broadcast communication H04H; communication systems using wireless links for non-selective communication, e.g. wireless extensions H04M1/72) - G05B: CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS (systems for controlling or regulating non-electric variables G05D; systems for regulating electric or magnetic variables G05F; control devices or systems insofar as characterised by mechanical features only G05G) ## Abstract An integrated accessory control system can integrate functionality (services) of multiple disparate accessories and provide a unified user interface for interacting with the system via a controller device. An integrated accessory control system can include one accessory that can detect an event or action and send a notification to the controller device and at least one other accessory, such as an Internet Protocol (IP) camera, that can be operated in response to the notification. In response to the notification, a controller device can generate an integrated user interface for interacting with the accessories in the integrated accessory control system. The interface can include a live feed from the IP camera, which can provide a media stream responsive to instructions from the controller. ## Claims ### Claim 1 A method comprising: receiving, at a controller device, a notification that an activator accessory in an automated environment has been activated, wherein the activator accessory is associated with an integrated accessory control system that comprises an activator accessory service provided by the activator accessory and further comprises one or more other accessory services, the one or more other accessory services comprising a video streaming service provided by an Internet Protocol camera, wherein the activator accessory and the Internet Protocol camera are separate devices; presenting, at the controller device, a first graphical user interface indicating that the activator accessory has been activated, the first graphical user interface comprising a selectable control element to initiate an interaction with the integrated accessory control system associated with the activator accessory; and in response to the user selecting the selectable control element: initiating, by the controller device, a video stream from the Internet Protocol camera to the controller device; and presenting, at the controller device, a second graphical user interface for interacting with the one or more accessory services of the integrated accessory control system, the second graphical user interface comprising a live view of the video stream. ### Claim 2 (depends on 1) The method of claim 1, wherein the one or more other accessory services associated with the integrated accessory control system further include a second accessory service and wherein the second graphical user interface provides a selectable control element to interact with the second accessory service. ### Claim 3 (depends on 1) The method of claim 1, wherein the second graphical user interface comprises a selectable control element to record the video stream. ### Claim 4 (depends on 1) The method of claim 1, wherein the second graphical user interface comprises a selectable control element to save a still image from the video stream. ### Claim 5 (depends on 1) The method of claim 1, wherein initiating the video stream comprises initiating recording of the video stream. ### Claim 6 (depends on 1) The method of claim 1, wherein the integrated accesory control system is an integrated entry control system and wherein the one or more other acessory services include a lock mechanism service usable to control a door lock. ### Claim 7 (depends on 6) The method of claim 6, wherein the activator accessory comprises a doorbell. ### Claim 8 (depends on 6) The method of claim 6, wherein the second graphical user interface comprises a selectable control element to lock or unlock the door lock. ### Claim 9 (depends on 1) The method of claim 1, wherein the controller communicates directly with the activator accessory and the Internet Protocol camera. ### Claim 10 (depends on 1) The method of claim 1, wherein the controller device communicates with at least one of the activator accessory or the Internet Protocol camera via a coordiantor device associated with the automated environment. ### Claim 11 (depends on 1) The method of claim 1, wherein the activator accessory comprises at least one of: a mechanical control operable to activate the activator accessory; a motion sensor that detects motion in an area where the motion sensor is installed, wherein the activator accessory is activated in response to detected motion; a movement sensor that detects movement of an object to which the movement sensor is attached, wherein the activator accessory is activated in response to detected movement; or a sound sensor, wherein the activator accessory is activated in response to a detected sound exceeding a background sound level. ### Claim 12 A computer readable storage medium having stored therein program code that, when executed by one or more processors of a controller device, cause the one or more processors to perform operations comprising: receiving, at a controller device, a notification that an activator accessory in an automated environment has been activated, wherein the activator accessory is associated with an integrated accessory control system that comprises an activator accessory service provided by the activator accessory and further comprises one or more other accessory services, the one or more other accessory services comprising a digital content streaming service provided by an Digital camera, wherein the activator accessory and the digital camera are separate devices; presenting, at the controller device, a first graphical user interface indicating that the activator accessory has been activated, the first graphical user interface comprising a selectable control element to initiate an interaction with the integrated accessory control system associated with the activator accessory; and in response to the user selecting the selectable control element: initiating, by the controller device, a digital content stream from the digital camera to the controller device; and presenting, at the controller device, a second graphical user interface for interacting with the one or more accessory services of the integrated accessory control system, the second graphical user interface comprising a live view of the digital content stream. ### Claim 13 (depends on 12) The computer readable storage medium of claim 12, wherein the one or more other accessory services associated with the integrated accessory control system further include a second accessory service and wherein the second graphical user interface provides a selectable control element to interact with the second accessory service. ### Claim 14 (depends on 12) The computer readable storage medium of claim 12, wherein the second graphical user interface comprises a selectable control element to record the digital content stream. ### Claim 15 (depends on 12) The computer readable storage medium of claim 12, wherein the second graphical user interface comprises a selectable control element to save an image from the digital content stream. ### Claim 16 (depends on 12) The computer readable storage medium of claim 12, wherein initiating the digital content stream comprises initiating recording of the digital content stream. ### Claim 17 (depends on 12) The computer readable storage medium of claim 12, wherein the integrated accessory control system is an integrated entry control system and wherein the one or more other accessory services include a lock mechanism service usable to control a door lock. ### Claim 18 A controller device, comprising: at least one memory to store information about one or more accessories; a wireless communication interface; and a processing subsystem coupled to the at least one memory and the wireless communication interface, the processing subsystem comprising one or more processors configured to: receive a notification that an activator accessory in an automated environment has been activated, wherein the activator accessory is associated with an integrated accessory control system that comprises an activator accessory service provided by the activator accessory and further comprises one or more other accessory services, the one or more other accessory services comprising a digital content streaming service provided by an Internet Protocol camera, wherein the activator accessory and the Internet Protocol camera are separate devices; presenting a first graphical user interface indicating that the activator accessory has been activated, the first graphical user interface comprising a selectable control element to initiate an interaction with the integrated accessory control system associated with the activator accessory; and in response to the user selecting the selectable control element: initiate a video stream from the Internet Protocol camera to the controller device; and present a second graphical user interface for interacting with the one or more accessory services of the integrated accessory control system, the second graphical user interface comprising a live view of the video stream. ### Claim 19 (depends on 18) The controller device of claim 18, wherein the one or more other accessory services associated with the integrated accessory control system further include a second accessory service and wherein the second graphical user interface provides a selectable control element to interact with the second accessory service. ### Claim 20 (depends on 19) The controller device of claim 19, wherein the activator accessory comprises at least one of: a mechanical control operable to activate the activator accessory; a motion sensor that detects motion in an area where the motion sensor is installed, wherein the activator accessory is activated in response to detected motion; a movement sensor that detects movement of an object to which the movement sensor is attached, wherein the activator accessory is activated in response to detected movement; or a sound sensor, wherein the activator accessory is activated in response to a detected sound exceeding a background sound level.