Method and system of controlling device using real-time indoor image

A device control system for controlling a device, the device control system comprising:

• a camera connected to a network;
• a display;
• an input interface;
• a processor;
• and
• a computer-readable storage medium storing instructions which, when executed by the processor, cause the processor to perform operations comprising:
• receiving an image captured by the camera connected to the network and displaying the image in real time on a touch screen;
• searching for a first device and a second device that are connected to the network and is controllable;
• displaying first guide information for guiding a user to designate a first setting zone corresponding to the first device that is displayed in the image;
• receiving a first user input within the image at a position of the first device in the image, and designating the first setting zone at the position of the first user input;
• displaying second guide information for guiding the user to designate a second setting zone corresponding to the second device that is not displayed in the image;
• receiving a second user input at a position at which the first device is not located, within the image;
• and
• based on a determination that the second device is not recognized in the image, designing the second setting zone at the position of the second user input that does not overlap with the position of the first device and positions of setting zones of other devices, and displaying the second setting zone in the image after designating the second setting zone;
• controlling the first device and the second device according to control inputs interacting with the first setting zone and the second setting zone, respectively;
• and
• updating the position of the first setting zone of the first device by changing the position of the first setting zone of the first device that is designated by the user to a different position within the image according to a change in the image in real time.