# US 2017/0287321 A1 · Display apparatus, system, and method for controlling an external device

> Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt

- Number: US 2017/0287321 A1
- Document ID: US20170287321A1
- Status: Published application
- Kind: A1
- Application number: 15/454,474
- Filed: 2017-03-09
- Published: 2017-10-05
- Earliest claimed: 2016-04-01
- Assignees: Samsung Electronics Co., Ltd.
- Applicants: Samsung Electronics Co., Ltd.
- Inventors: Woo-ram ANN; Cheul-hee HAHM; Tae-young LEE; Joo-young JUNG; Ki-hyeok KOO
- Web page: https://patentgrep.com/patent/US20170287321A1

## CPC

- G08C 17/02: using a radio link
- H04L 12/2814: {Exchanging control software or macros for controlling appliance services in a home automation network (arrangements for maintenance or administration involving configuration of the network and network elements H04L41/08)}
- H04L 12/282: {based on user interaction within the home (receiver circuitry for displaying additional information being controlled by a remote control apparatus H04N21/42204)}
- H04L 12/4625: {Single bridge functionality, e.g. connection of two networks over a single bridge}
- H04N 21/41265: {having a remote control device for bidirectional communication between the remote control device and client device (unidirectional remote control device H04N21/42204)}
- H04N 21/4722: for requesting additional data associated with the content
- G06F 18/22: Matching criteria, e.g. proximity measures
- G08C 2201/30: User interface
- G08C 2201/70: Device selection
- G08C 2201/92: Universal remote control
- G08C: TRANSMISSION SYSTEMS FOR MEASURED VALUES, CONTROL OR SIMILAR SIGNALS (fluid pressure transmitting systems F15B; mechanical means for transferring the output of a sensing member into a different variable G01D5/00; mechanical control systems G05G)
- H04L: TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION (arrangements common to telegraphic and telephonic communication H04M)
- H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION
- G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N)

## Abstract

A display apparatus is provided. The display apparatus is configured to control at least one external device to which an identification marker is attached. The display apparatus also includes an imaging sensor configured to obtain an image of the at least one external device, and a communicator configured to perform communication with at least one of a server and the at least one external device. The display apparatus also includes a processor configured to transmit identification information obtained from an identification marker included in the image, and in response to control information for controlling the at least one external device being received from the server or the at least one external device, control the display to match and display an object for controlling the at least one external device with an external device image based on the control information.

## Claims

### Claim 1

A display apparatus configured to control at least one external device to which an identification marker is attached, the apparatus comprising:
a display;
an imaging sensor configured to obtain an image of the at least one external device;
a communicator configured to perform communication with at least one of a server and the at least one external device; and
a processor configured to transmit identification information obtained from an identification marker included in the image, and in response to control information for controlling the at least one external device being received from the server or the at least one external device, control the display to match and display an object for controlling the at least one external device with an external device image based on the control information.

### Claim 2 (depends on 1)

The apparatus as claimed in claim 1, wherein the control information includes at least one of a current state of the at least one external device, a control parameter of the at least one external device, a title of the at least one external device, software for controlling the at least one external device, and information associated with the object for controlling the at least one external device.

### Claim 3 (depends on 2)

The apparatus as claimed in claim 2, wherein the control parameter of the at least one external device includes at least one of a type of control function, a control range, and control accuracy for each type.

### Claim 4 (depends on 2)

The apparatus as claimed in claim 2, wherein the information associated with the object includes at least one of a switch image for controlling the at least one external device, a button image for controlling the at least one external device, and a remote control image for controlling the at least one external device.

### Claim 5 (depends on 3)

The apparatus as claimed in claim 3, further comprising:
an input unit configured to receive a user input,
wherein the processor, in response to the user input regarding the object being received, generates a control command corresponding to the user input based on the control parameter of the at least one external device and transmits the generated control command to the at least one external device.

### Claim 6 (depends on 5)

The apparatus as claimed in claim 5, wherein the object is an image operable to respond to the user input,
wherein the processor generates the control command corresponding to an operation state of the object and transmits the generated control command to the at least one external device.

### Claim 7 (depends on 1)

The apparatus as claimed in claim 1, wherein the processor generates and causes a user interface screen to be displayed for controlling and managing the at least one external device with which the object is matched.

### Claim 8

A system including a server and a display apparatus, the system comprising:
a display apparatus configured to obtain an image of a plurality of external devices to which an identification marker is attached to each respective external device of the plurality of external devices, and transmit identification information obtained from a plurality of identification markers included in the image; and
a server configured to receive the identification information, and in response, transmit control information for controlling each respective external device of the plurality of external devices to the display apparatus,
wherein the display apparatus matches each respective external device of the plurality of external devices with a corresponding external device image, and displays a plurality of objects for controlling each respective external device of the plurality of external devices wherein each object of the plurality of objects comprises the corresponding external device image included in the control information.

### Claim 9

A controlling method of a display apparatus for controlling at least one external device to which an identification marker is attached, the method comprising:
obtaining an image of the at least one external device;
transmitting identification information obtained from an identification marker included in the image to a server, and receiving control information for controlling the at least one external device from the server; and
matching and displaying an object for controlling the at least one external device with an external device image based on the control information received from the server.

### Claim 10 (depends on 9)

The method as claimed in claim 9, wherein the control information includes at least one of a current state of the at least one external device, a control parameter of the at least one external device, a title of the at least one external device, software for controlling the at least one external device, and information on an object for controlling the at least one external device.

### Claim 11 (depends on 10)

The method as claimed in claim 10, wherein the control parameter of the at least one external device includes at least one of a type of control function, a control range, and control accuracy for each type.

### Claim 12 (depends on 10)

The method as claimed in claim 10, wherein the object for controlling the at least one external device comprises at least one of a switch image for controlling the at least one external device, a button image for controlling the at least one external device, and a remote control image for controlling the at least one external device.

### Claim 13 (depends on 11)

The method as claimed in claim 11, further comprising:
receiving a user input directed to the object; and
generating a control command corresponding to the user input based on the control parameter for the at least one external device corresponding to the object, and transmitting the generated control command to the at least one external device.

### Claim 14 (depends on 13)

The method as claimed in claim 13, wherein the object is an image operable to respond to the user input,
wherein the transmitting comprises generating the control command corresponding to an operation state of the object and transmitting the generated control command to the at least one external device.

### Claim 15 (depends on 9)

The method as claimed in claim 9, wherein the displaying comprises generating and displaying a user interface screen for controlling and managing the at least one external device with which the object is matched.
