# US 2015/0145786 A1 · Method of controlling electronic device using transparent display and apparatus using the same > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2015/0145786 A1 - Document ID: US20150145786A1 - Status: Published application - Kind: A1 - Application number: 14/319,115 - Filed: 2014-06-30 - Published: 2015-05-28 - Earliest claimed: 2013-11-27 - Assignees: Electronics and Telecommunications Research Institute - Applicants: Electronics and Telecommunications Research Institute - Inventors: Jeong Mook LIM; Hee Sook SHIN; Jong Uk LEE - Web page: https://patentgrep.com/patent/US20150145786A1 ## CPC - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - G06F 1/1637: {Details related to the display arrangement, including those related to the mounting of the display in the housing} - G06F 1/1643: {the display being associated to a digitizer, e.g. laptops that can be used as penpads (details related to the relative motion of the display enclosure with respect to the body enclosure, e.g. to move between laptop and tablet PC configuration G06F1/1615)} - G06F 1/1686: {the I/O peripheral being an integrated camera} - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 16/5838: {using colour} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Provided are an electronic device control apparatus and method using a transparent display. The electronic device control apparatus includes a first image acquirer configured to acquire a plurality of thing-direction images respectively captured by a plurality of cameras, a virtual image generator configured to combine the plurality of thing-direction images to generate a virtual image corresponding to a thing-direction region projected on the transparent display, a user interface configured to receive a user input, and a function attribute mapper configured to recognize an object included in the virtual image, based on the user input, and map, to the virtual image, a function attribute for controlling the recognized object. ## Claims ### Claim 1 An electronic device control apparatus using a transparent display, the electronic device control apparatus comprising: a first image acquirer configured to acquire a plurality of thing-direction images respectively captured by a plurality of cameras; a virtual image generator configured to combine the plurality of thing-direction images to generate a virtual image corresponding to a thing-direction region projected on the transparent display; a user interface configured to receive a user input; and a function attribute mapper configured to recognize an object included in the virtual image, based on the user input, and map, to the virtual image, a function attribute for controlling the recognized object. ### Claim 2 (depends on 1) The electronic device control apparatus of claim 1, wherein the virtual image generator generates the virtual image having the same size as a size of an image corresponding to the thing-direction region projected on the transparent display. ### Claim 3 (depends on 1) The electronic device control apparatus of claim 1, wherein the virtual image generator searches for an overlapping region between a first thing-direction image and a second thing-direction image of the plurality of thing-direction images, and generates the virtual image which is obtained by overlapping the first thing-direction image and the second thing-direction image by a size of the overlapping region. ### Claim 4 (depends on 1) The electronic device control apparatus of claim 1, wherein the virtual image generator is activated to operate when a sum of region sizes of the plurality of thing-direction images is greater than a size of the transparent display. ### Claim 5 (depends on 1) The electronic device control apparatus of claim 1, further comprising: a second image acquirer configured to acquire a plurality of user-direction images respectively captured by the plurality of cameras; and a camera driver configured to trace a user's look direction in the plurality of user-direction images, and change the photographing directions of the plurality of cameras so that an angle between the look direction and each of the photographing directions of the plurality of cameras is within a predetermined range. ### Claim 6 (depends on 1) The electronic device control apparatus of claim 1, wherein the user interface provides the user with a feedback on a result of the user input or an event which occurs in the electronic device control apparatus using the transparent display. ### Claim 7 (depends on 1) The electronic device control apparatus of claim 1, wherein the function attribute mapper transmits, as a query, a partial region of the virtual image selected based on the user input to an image search engine, and searches for attribute information about an object recognized by using an object recognition result transferred from the image search engine. ### Claim 8 (depends on 7) The electronic device control apparatus of claim 7, wherein the image search engine is disposed remotely from the electronic device control apparatus using the transparent display, and transmits and receives data over a wireless communication network. ### Claim 9 (depends on 1) The electronic device control apparatus of claim 1, wherein the function attribute mapper maps the function attribute to a corresponding region, which is set by changing a position, in the virtual image. ### Claim 10 (depends on 1) The electronic device control apparatus of claim 1, wherein a physical central position of the transparent display matches a center of the virtual image. ### Claim 11 An electronic device control method using a transparent display, the electronic device control method comprising: acquiring a plurality of thing-direction images respectively captured by a plurality of cameras; combining the plurality of thing-direction images to generate a virtual image corresponding to a thing-direction region projected on the transparent display; and recognizing an object included in the virtual image, based on a user input, and mapping, to the virtual image, a function attribute for controlling the recognized object. ### Claim 12 (depends on 11) The electronic device control method of claim 11, wherein the generating of a virtual image comprises searching for an overlapping region between a first thing-direction image and a second thing-direction image of the plurality of thing-direction images, and generating the virtual image which is obtained by overlapping the first thing-direction image and the second thing-direction image by a size of the overlapping region. ### Claim 13 (depends on 11) The electronic device control method of claim 11, wherein the generating of a virtual image comprises generating the virtual image having the same size as a size of an image corresponding to the thing-direction region projected on the transparent display. ### Claim 14 (depends on 11) The electronic device control method of claim 11, wherein the generating of a virtual image comprises generating the virtual image having the same center as a physical central position of the transparent display. ### Claim 15 (depends on 11) The electronic device control method of claim 11, wherein the generating of a virtual image comprises: searching for an overlapping region between a first thing-direction image and a second thing-direction image of the plurality of thing-direction images, and generating an auxiliary virtual image which is obtained by overlapping the first thing-direction image and the second thing-direction image by a size of the overlapping region; and generating, as a final virtual image (a cropped image), an image which is obtained by cropping a region other than a region occupied by the transparent display in the auxiliary virtual image. ### Claim 16 (depends on 11) The electronic device control method of claim 11, further comprising: before the generating of a virtual image, acquiring a plurality of user-direction images respectively captured by the plurality of cameras; and tracing a user's look direction in the plurality of user-direction images, and changing the photographing directions of the plurality of cameras so that an angle between the look direction and each of the photographing directions of the plurality of cameras is within a predetermined range. ### Claim 17 (depends on 16) The electronic device control method of claim 16, further comprising performing activation when a sum of region sizes of the plurality of thing-direction images is greater than a size of the transparent display. ### Claim 18 (depends on 11) The electronic device control method of claim 11, wherein the mapping of a function attribute comprises: transmitting, as a query, a partial region of the virtual image selected based on the user input to an image search engine; and searching for attribute information about an object recognized by using an object recognition result transferred from the image search engine. ### Claim 19 (depends on 11) The electronic device control method of claim 11, wherein the mapping of a function attribute comprises mapping the function attribute to a corresponding region, which is set by changing a position, in the virtual image. ### Claim 20 (depends on 18) The electronic device control method of claim 18, wherein the image search engine is disposed remotely from the electronic device control apparatus using the transparent display, and transmits and receives data over a wireless communication network.