# US 12,001,604 B2 · Information processing device, information processing system, information processing method, and program > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,001,604 B2 - Document ID: US12001604B2 - Status: Granted - Kind: B2 - Application number: 17/281,255 - Filed: 2019-09-19 - Published: 2022-07-14 - Granted: 2024-06-04 - Earliest claimed: 2018-11-07 - Assignees: Sony Corporation - Applicants: Sony Corporation - Inventors: Ryohei Yasuda; Shinichi Kawano - Web page: https://patentgrep.com/patent/US12001604B2 ## CPC - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - G06F 3/0482: Interaction with lists of selectable items, e.g. menus - G06V 40/20: Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16) - H04N 1/32101: {Display, printing, storage or transmission of additional information, e.g. ID code, date and time or title} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06V: IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract An image analysis unit that detects a user's point of gaze with respect to display information of a display unit on the basis of an image captured by a camera, a user attention information analysis unit that extracts attention information on which a user's point of gaze had been for a prescribed length of time or longer and generates a list including information corresponding to the attention information, and a display information generation unit that generates a display list including list configuration data. The user attention information analysis unit continuously inputs user's point-of-gaze information, and executes sorting processing for setting, in the top of the list, latest attention information or attention information on which the user's point of gaze is for a long length of time. The display information generation unit generates a display list including top data of list configuration data, for example. ## Claims ### Claim 1 An information processing device comprising: circuitry configured to detect a user's point of gaze with respect to display information of a display, the display information being a message in a chat screen, extract attention information of the user on a basis of an analysis of the user's point of gaze, generate a list including information corresponding to the attention information, generate a display list including configuration data of the list, wherein the circuitry is configured to detect a message gazed by a user, extract an attention message on which the user's point of gaze had been for a prescribed length of time or longer, generate a user ID list including a user ID of an originator of the attention message, and generate a user ID selection menu including individually selectable user IDs. ### Claim 2 (depends on 1) The information processing device according to claim 1, wherein the circuitry is configured to detect a user's point of gaze with respect to display information of the display on a basis of an image captured by a camera, and the circuitry is configured to extract attention information on which the user's point of gaze had been for a prescribed length of time or longer, and generates a list including information corresponding to the attention information. ### Claim 3 (depends on 1) The information processing device according to claim 1, wherein the circuitry is configured to continuously input user's point-of-gaze information, and, on a basis of input information, executes list update processing accompanied by sorting processing for setting latest attention information in a top of the list. ### Claim 4 (depends on 1) The information processing device according to claim 1, wherein the circuitry is configured to continuously input user's point-of-gaze information, and, on a basis of input information, execute list update processing accompanied by sorting processing for setting, in a top of the list, the extracted attention information on which a user's point of gaze is for a long length of time among two or more sets of the extracted attention information on which the user's point of gaze had been for a prescribed length of time or longer. ### Claim 5 (depends on 1) The information processing device according to claim 1, wherein the circuitry is configured to generate a display list including only highly ranked data of configuration data of the list. ### Claim 6 (depends on 1) The information processing device according to claim 1, wherein the circuitry is configured to display the user ID selection menu on a user terminal, wherein the user ID selection menu has a configuration in which user ID entry is possible by user selection. ### Claim 7 (depends on 1) The information processing device according to claim 1, wherein the user ID selection menu includes user ID and a part of message. ### Claim 8 (depends on 1) The information processing device according to claim 1, wherein the display information is shop/restaurant information, the circuitry is configured to detect shop/restaurant information of a user's target of gaze, the circuitry is configured to extract attention shop/restaurant information on which the user's point of gaze had been for a prescribed length of time or longer, and generate a shop/restaurant list including a shop/restaurant name corresponding to the attention shop/restaurant information, and the circuitry is configured to generate a display list including a shop/restaurant name included in a shop/restaurant list. ### Claim 9 (depends on 8) The information processing device according to claim 8, wherein the circuitry is configured to generate a display list in which access information of a shop/restaurant name providing detailed information of each shop/restaurant is embedded in a shop/restaurant name included in the display list. ### Claim 10 A non-transitory computer-readable storage medium including computer executable instructions, wherein the instructions, when executed by an information processing device, cause the information processing device to perform a method, the method comprising: detecting a user's point of gaze with respect to display information of a display, the display information being a message in a chat screen; extracting attention information of the user on a basis of an analysis of the user's point of gaze; generating a list including information corresponding to the attention information; and generating a display list including configuration data of the list, wherein a message gazed by a user is detected, wherein an attention message on which the user's point of gaze had been for a prescribed length of time or longer is extracted, wherein a user ID list including a user ID of an originator of the attention message is generated, and wherein a user ID selection menu including individually selectable user IDs is generated. ### Claim 11 (depends on 10) The non-transitory computer-readable storage medium according to claim 10, wherein the method further comprising: extracting user's point of gaze with respect to display information of the display is extracted on a basis of an image captured by a camera; extracting attention information on which the user's point of gaze had been for a prescribed length of time or longer; and generating a list including information corresponding to the attention information. ### Claim 12 (depends on 10) The non-transitory computer-readable storage medium according to claim 10, wherein the method further comprising: continuously inputting user's point-of-gaze information; and executing, on a basis of input information, list update processing accompanied by sorting processing for setting latest attention information in a top of the list. ### Claim 13 (depends on 10) The non-transitory computer-readable storage medium according to claim 10, wherein the method further comprising: continuously inputting user's point-of-gaze information; and executing, on a basis of input information, execute list update processing accompanied by sorting processing for setting, in a top of the list, the extracted attention information on which a user's point of gaze is for a long length of time among two or more sets of the extracted attention information on which the user's point of gaze had been for a prescribed length of time or longer. ### Claim 14 (depends on 10) The non-transitory computer-readable storage medium according to claim 10, wherein the method further comprising: generating a display list including only highly ranked data of configuration data of the list. ### Claim 15 (depends on 10) The non-transitory computer-readable storage medium according to claim 10, wherein the method further comprising: displaying user ID selection menu on a user terminal, wherein the user ID selection menu has a configuration in which user ID entry is possible by user selection. ### Claim 16 (depends on 10) The non-transitory computer-readable storage medium according to claim 10, wherein the user ID selection menu includes user ID and a part of message. ### Claim 17 (depends on 10) The non-transitory computer-readable storage medium according to claim 10, wherein the display information is shop/restaurant information, and wherein the method further comprising: detecting shop/restaurant information of a user's target of gaze; extracting attention shop/restaurant information on which the user's point of gaze had been for a prescribed length of time or longer; generating a shop/restaurant list including a shop/restaurant name corresponding to the attention shop/restaurant information; and generating a display list including a shop/restaurant name included in a shop/restaurant list. ### Claim 18 (depends on 17) The non-transitory computer-readable storage medium according to claim 17, wherein the method further comprising: generating a display list in which access information of a shop/restaurant name providing detailed information of each shop/restaurant is embedded in a shop/restaurant name included in the display list. ### Claim 19 An information processing device comprising: circuitry configured to detect a user's point of gaze with respect to display information of a display, the display information being shop/restaurant information, extract attention information of the user on a basis of an analysis of the user's point of gaze, generate a list including information corresponding to the attention information, and generate a display list including configuration data of the list, wherein the circuitry is configured to detect shop/restaurant information of a user's target of gaze, extract attention shop/restaurant information on which the user's point of gaze had been for a prescribed length of time or longer, generate a shop/restaurant list including a shop/restaurant name corresponding to the attention shop/restaurant information, and generate a display list including a shop/restaurant name included in a shop/restaurant list. ### Claim 20 (depends on 19) The information processing device according to claim 19, wherein the circuitry is configured to generate a display list in which access information of a shop/restaurant name providing detailed information of each shop/restaurant is embedded in a shop/restaurant name included in the display list.