# US 10,372,297 B2 · Image control method and device > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,372,297 B2 - Document ID: US10372297B2 - Status: Granted - Kind: B2 - Application number: 15/585,325 - Filed: 2017-05-03 - Published: 2017-11-16 - Granted: 2019-08-06 - Earliest claimed: 2016-05-11 - Assignees: Fujitsu Limited - Applicants: Fujitsu Limited - Inventors: Soichi Nishiyama; Hiroaki Sugimura; Ken Fujiki; Hajime Hirozumi - Web page: https://patentgrep.com/patent/US10372297B2 ## CPC - G06F 3/0482: Interaction with lists of selectable items, e.g. menus - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) - G06F 3/1454: {involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay} - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - G06F 16/51: Indexing; Data structures therefor; Storage structures - G06Q 10/101: Collaborative creation, e.g. joint development of products or services - G06Q 10/109: Time management, e.g. calendars, reminders, meetings or time accounting - G06Q 30/0201: Market modelling; Market analysis; Collecting market data - G06Q 30/0203: Market surveys; Market polls - G06T 11/26: Drawing of charts or graphs - G06T 2200/24: involving graphical user interfaces [GUIs] - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - G06Q: INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR - G06T: IMAGE DATA PROCESSING OR GENERATION, IN GENERAL ## Abstract An image control method includes referencing a first storage section where selection option information, for which a selection operation is received on a display device, is stored in association with a selection timing, and a second storage section where detection information, detected by a detection device that detects at least one of a state of a area at which the display device is disposed or a state of a user, is stored in association with detection time information; and identifying an item of selection option information stored at the first storage section in association with a timing that corresponds to the time information stored at the second storage section in association with detection information indicating a predetermined state; and effecting control so as to display the identified item of selection option information on the display device using a different display mode from that of other displayed selection option information. ## Claims ### Claim 1 An image control method, comprising: receiving, using a display device displaying a plurality of items of selection option information, a selection operation by a user of an item of the selection option information, and storing, in a first storage section, the selected item of the selection option information in association with a timing at which the selection operation for the selection option information is receive; detecting, using a detection device, detection information indicating at least one of a state of an area at which the display device is disposed or a state of a user present in the area at which the display device is disposed, and storing, in a second storage section, the detection information in association with time information indicating a time at which the detection information is detected; extracting, by a processor, from the second storage section, time information that is associated with detection information indicating that at least one of the area or the user is in a predetermined state; identifying, by the processor, an item of the selection option information stored at the first storage section in association with a timing that corresponds to the time information stored at the second storage section; associating, by the processor, a change timing with the identified item of the selection option information, and storing the identified item of the selection option information and the associated change timing in a third storage section; determining, by the processor, whether the change timing of any of the plurality of items of the selection option information displayed on the display device has arrived, by referencing the third storage section; and effecting control by the processor so as to display, on the display device, the item of the selection option information having the change timing associated therewith determined to have arrived, using a different display mode from that of other displayed items of the selection option information. ### Claim 2 (depends on 1) The image control method of claim 1, wherein the detection information includes at least one of acoustic pressure in the area, a pulse rate of the user, speech content of the user, movement of the user, or position information on the user. ### Claim 3 (depends on 2) The image control method of claim 2, further comprising, when identifying the item of the selection option information: in a case in which the detection information is the acoustic pressure in the area, identifying an item of the selection option information for which a selection operation is received at a timing corresponding to time information indicating a time at which acoustic pressure of a specific value or greater is detected; in a case in which the detection information is the pulse rate of the user, identifying an item of the selection option information for which a selection operation is received at a timing corresponding to time information indicating a time at which a pulse rate of a specific value or greater is detected; in a case in which the detection information is the speech content of the user, identifying an item of the selection option information for which a selection operation is received at a timing corresponding to time information indicating a time at which speech containing a predetermined keyword is detected; in a case in which the detection information is the movement of the user, identifying an item of the selection option information for which a selection operation is received at a timing corresponding to time information indicating a time at which movement greater than a predetermined magnitude is detected; and in a case in which the detection information is the position information on the user, identifying an item of the selection option information for which a selection operation is received at a timing corresponding to time information indicating a time at which a user position, movement path, or movement speed satisfying a specific condition is detected. ### Claim 4 (depends on 1) The image control method of claim 1, wherein the item of the selection option information displayed using a different display mode from that of the other displayed items of the selection option information is displayed using a display mode that has greater visibility than that of the other displayed items of the selection option information. ### Claim 5 (depends on 1) The image control method of claim 1, wherein: a goal of a workshop held using the plurality of items of selection option information, or an attribute of a user group holding the workshop, is stored respectively in association with the selection option information and the timing stored at the first storage section, and in association with the detection information and the time information stored at the second storage section; and when displaying the plurality of items of selection option information on the display device, the selection option information and the timing stored at the first storage section in association with the goal of the workshop held using the plurality of items of selection option information, or in association with the user group holding the workshop, and the detection information and the time information stored at the second storage section, are referenced to identify the item of the selection option information to be displayed using a different display mode from that of the other displayed items of the selection option information. ### Claim 6 (depends on 1) The image control method of claim 1, wherein, in a case in which there is a plurality of items of the selection option information to be displayed using a different display mode from that of the other displayed items of the selection option information: a timing that makes the display mode differ from that of the other displayed items of the selection option information is set so as to be the same timing for each of the plurality of items of the selection option information to be displayed using a different display mode; or timings are set so as to differ from each other according to the timings at which selection operations are respectively received for the plurality of items of the selection option information to be displayed using a different display mode. ### Claim 7 (depends on 1) The image control method of claim 1, further comprising displaying a graphic representation of a relationship between the detection information and the time information on a display device. ### Claim 8 (depends on 1) The image control method of claim 1, wherein the other displayed items of the selection option information are respectively displayed so as to move in an up-down direction or a left-right direction on the display device, and the item of the selection option information displayed using a different display mode from that of the other displayed items of the selection option information is displayed in a stationary state, is displayed so as to move in an opposite direction to the direction in which the other displayed items of the selection option information are moving, or is displayed so as to move in a direction intersecting the direction in which the other displayed items of the selection option information are moving. ### Claim 9 (depends on 1) The image control method of claim 1, further comprising displaying, on another display device, the selection option information for which a selection operation on the display device is received. ### Claim 10 An image control device, comprising: a memory; and a processor coupled to the memory, the processor configured to: receive, using a display device displaying a plurality of items of selection option information, a selection operation by a user of an item of the selection option information, and store, in a first storage section, the selected item of the selection option information in association with a timing at which the selection operation for the selection option information is received; detect, using a detection device, detection information indicating at least one of a state of an area at which the display device is disposed or a state of a user present in the area at which the display device is disposed, and store, in a second storage section, the detection information in association with time information indicating a time at which the detection information is detected; extract, from the second storage section, time information that is associated with detection information indicating that at least one of the area or the user is in a predetermined state; identify an item of the selection option information stored at the first storage section in association with a timing that corresponds to the extracted time information from the second storage section; associate a change timing with the identified item of the selection option information, and store the identified item of the selection option information and the associated change timing in a third storage section; determine whether the change timing of any of the plurality of items of the selection option information displayed on the display device has arrived, by referencing the third storage section: and display, on the display device, the item of the selection option information having the change timing associated therewith determined to have arrived, using a different display mode from that of other displayed items of the selection option information. ### Claim 11 (depends on 10) The image control device of claim 10, wherein the detection information includes at least one of acoustic pressure in the area, a pulse rate of the user, speech content of the user, movement of the user, or position information on the user. ### Claim 12 (depends on 10) The image control device of claim 10, wherein the item of the selection option information displayed using a different display mode from that of the other displayed items of the selection option information is displayed using a display mode that has greater visibility than that of the other displayed items of the selection option information. ### Claim 13 (depends on 10) The image control device of claim 10, wherein: a goal of a workshop held using the plurality of items of selection option information, or an attribute of a user group holding the workshop, is stored respectively in association with the selection option information and the timing stored at the first storage section, and in association with the detection information and the time information stored at the second storage section; and the image control device further comprises an identification section that, when displaying the plurality of items of selection option information on the display device, references the selection option information and the timing stored at the first storage section in association with the goal of the workshop held using the plurality of items of selection option information, or in association with the user group holding the workshop, and the detection information and the time information stored at the second storage section, to identify the item of the selection option information to be displayed using a different display mode from that of the other displayed items of the selection option information. ### Claim 14 (depends on 10) The image control device of claim 10, wherein, in a case in which there is a plurality of items of the selection option information to be displayed using a different display mode from that of the other displayed items of the selection option information: a timing that makes the display mode differ from that of the other displayed items of the selection option information is set so as to be the same timing for each of the plurality of items of the selection option information to be displayed using a different display mode, or timings are set so as to differ from each other according to the timings at which selection operations are respectively received for the plurality of items of the selection option information to be displayed using a different display mode. ### Claim 15 (depends on 10) The image control device of claim 10, wherein the processor is further configured to display a graphic representation of a relationship between the detection information and the time information on a display device. ### Claim 16 (depends on 10) The image control device of claim 10, wherein the processor is further configured to display, on another display device, the selection option information for which a selection operation on the display device is received. ### Claim 17 An image control system comprising: a first display device that displays a plurality of items of selection option information and receives a selection operation by a user of an item of the selection option information; a management device including: a memory; and a processor coupled to the memory, the processor being configured to: store, in a first storage section, the selected item of the selection option information in association with a timing at which the selection operation for the selection option information is received; detect, using a detection device, detection information indicating at least one of a state of an area at which the first display device is disposed or a state of a user present in the area at which the first display device is disposed, and store, in a second storage section, the detection information in association with time information indicating a time at which the detection information is detected; extract, from the second storage section, time information that is associated with detection information indicating that at least one of the area or the user is in a predetermined state; identify an item of the selection option information stored at the first storage section in association with a timing that corresponds to the extracted time information from the second storage section; associate a change timing with the identified item of the selection option information, and store the identified item of the selection option information and the associated change timing in a third storage section; and determine whether the change timing of any of the plurality of items of the selection option information displayed on the display device has arrived, by referencing the third storage section; and a first control device including: a memory; and a processor coupled to the memory of the first control device, the processor being configured to display, on the first display device, the item of selection option information having the change timing associated therewith determined by the management device to have arrived, using a different display mode from that of other displayed items of the selection option information. ### Claim 18 (depends on 17) The image control system of claim 17, further comprising: a second display device; and a second control device, wherein: the first control device forwards, to the second control device, the selection option information for which the selection operation is received on the first display device, and the second control device includes: a memory; and a processor coupled to the memory, the processor configured to effect control such that the forwarded selection option information is displayed on the second display device. ### Claim 19 (depends on 18) The image control system of claim 18, further comprising: a third display device; and a third control device, wherein: the first control device forwards the detection information and the time information to the third control device, and the third control device includes: a memory; and a processor coupled to the memory, the processor configured to effect control such that a graphic representation of the relationship between the forwarded detection information and the time information is displayed on the third display device.