# US 2021/0279961 A1 · Information providing device, terminal device, display system, storage medium, and information providing method > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 2021/0279961 A1 - Document ID: US20210279961A1 - Status: Published application - Kind: A1 - Application number: 16/486,060 - Filed: 2017-12-01 - Published: 2021-09-09 - Earliest claimed: 2017-03-31 - Assignees: Sharp Kabushiki Kaisha - Applicants: Sharp Kabushiki Kaisha - Inventors: AKIRA WATANABE - Web page: https://patentgrep.com/patent/US20210279961A1 ## CPC - G06T 19/006: {Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00)} - G06F 3/0481: based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance - G06F 3/16: Sound input; Sound output (speech processing G10L) - G06F 13/00: Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units (interface circuits for specific input/output devices G06F3/00 {; multiprogram control therefor G06F9/46}; multiprocessor systems G06F15/16 ) - G06F 18/24: Classification techniques - G06T 13/40: of characters, e.g. humans, animals or virtual beings - G06V 20/10: Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) - G06V 20/20: in augmented reality scenes - G09G 5/00: Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators (image data processing or generation, in general G06T) - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - G09G: ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION (arrangements for transferring data between digital computers and displays G06F3/14; static indicating arrangements comprising an association of a number of separate sources or light control cells G09F9/00; static indicating arrangements comprising integral associations of a number of light sources H01J, H01K, H10H, H05B33/12; scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission, details thereof H04N1/00) ## Abstract The server control section of the server carries out control of at least either referring to device identification information and determining character management information, and transmitting the determined character management information to the terminal device, or referring to character management information and determining device management information, and transmitting the determined device management information to the household electric appliance. ## Claims ### Claim 1 An information providing device for providing information to a terminal device configured to superimpose an image of a character over a captured image, the information providing device comprising: a communication section; and a control section, the control section being configured to carry out control of at least either (i) referring to device identification information for identifying an individual device shown in the captured image and (ii) determining character management information for managing a form of presenting the character, and transmitting the determined character management information to the terminal device via the communication section, or referring to the character management information and determining device management information for determining a state that the individual device presents, and transmitting the device management information to the individual device via the communication section. ### Claim 2 (depends on 1) The information providing device according to claim 1, wherein the control section is configured to carry out control of at least either referring to the device identification information and updating the character management information, or referring to the character management information and updating the device management information. ### Claim 3 (depends on 1) The information providing device according to claim 1, wherein the control section is configured to select content of speech by the character with reference to (i) terminal identification information for distinguishing between the terminal device and another terminal device and (ii) character related information obtainable from a history of conversation between the character and a user. ### Claim 4 (depends on 1) The information providing device according to claim 1, wherein the control section is configured to, in accordance with device operation information, change the form of presenting the character, the device operation information being information related to operation of the individual device shown in the captured image. ### Claim 5 (depends on 1) The information providing device according to claim 1, wherein the device identification information includes device classification information, which is classification information obtainable from appearance of the individual device shown in the captured image. ### Claim 6 (depends on 5) The information providing device according to claim 5, wherein the device identification information further includes capturing position information of the captured image. ### Claim 7 (depends on 5) The information providing device according to claim 5, wherein the device identification information further includes unique information obtainable through communication with the individual device shown in the captured image. ### Claim 8 A terminal device, comprising: an image capturing section; a presenting section; a control section; and a communication section, the presenting section including a display section, the control section being configured to carry out control of at least either (i) obtaining, from an information providing device for providing information to the terminal device, character management information for managing a form of presenting a character which character management information has been determined with reference to device identification information for identifying an individual device shown in a captured image and (ii) causing the display section to superimpose an image of the character over the captured image captured by the image capturing section, or transmitting the device identification information for identifying the individual device to the information providing device via the communication section. ### Claim 9 (depends on 8) The terminal device according to claim 8, wherein the device identification information is associated with the character management information, which is referred to for determining device management information for determining a state that the individual device presents. ### Claim 10 (depends on 9) The terminal device according to claim 9, wherein the control section is configured to cause the presenting section to output content of speech by the character which content has been selected with reference to (i) terminal identification information for distinguishing between the terminal device and another terminal device and (ii) character related information obtainable from a history of conversation between the character and a user. ### Claim 11 (depends on 9) The terminal device according to claim 9, wherein control section is configured to (i) refer to the form of presenting the character which form has been changed in accordance with device operation information, which is information related to operation of the individual device shown in the captured image, and (ii) cause the display section to superimpose the image of the character over the captured image captured by the image capturing section. ### Claim 12 A display system, comprising: an information providing device; and a terminal device, terminal device including: an image capturing section; a presenting section; a terminal control section; and a terminal communication section, the presenting section including a display section, the information providing device including: a server communication section; and a server control section, the server control section being configured to carry out control of at least either (i) referring to device identification information for identifying an individual device shown in a captured image and (ii) determining character management information for managing a form of presenting a character, and transmitting the determined character management information to the terminal device via the server communication section, or referring to the character management information and determining device management information for determining a state that the individual device presents, and transmitting the determined device management information to the individual device via the server communication section, the terminal control section being configured to refer to the character management information and cause the display section to superimpose an image of the character over the captured image captured by the image capturing section. ### Claim 13 (depends on 1) A computer readable non-transitory storage medium that stores a program for causing a computer to function as an information providing device according to claim 1, the program causing the computer to function as the control section. ### Claim 14 (depends on 9) A computer readable non-transitory storage medium that stores a program for causing a computer to function as a terminal device according to claim 9, the program causing the computer to function as the control section. ### Claim 15 An information providing method for providing information to a terminal device configured to superimpose an image of a character over a captured image, the information providing method comprising at least either: (i) referring to device identification information for identifying an individual device shown in the captured image and (ii) determining character management information for managing a form of presenting the character, and transmitting the determined character management information to the terminal device, or referring to the character management information and determining device management information for determining a state that the individual device presents, and transmitting the determined device management information to the individual device.