# US 10,503,373 B2 · Visual feedback for highlight-driven gesture user interfaces > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,503,373 B2 - Document ID: US10503373B2 - Status: Granted - Kind: B2 - Application number: 13/420,275 - Filed: 2012-03-14 - Published: 2013-09-19 - Granted: 2019-12-10 - Earliest claimed: 2012-03-14 - Assignees: Sony Interactive Entertainment Inc. - Applicants: Carsten Schwesig; Marvin Demerchant; Steven Friedlander; Brian Johnson; Sebrina Yeh; Hyehoon Yi; David Young - Inventors: Carsten Schwesig; Marvin Demerchant; Steven Friedlander; Brian Johnson; Sebrina Yeh; Hyehoon Yi; David Young - Web page: https://patentgrep.com/patent/US10503373B2 ## CPC - G06F 3/04842: Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 3/0304: {Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042)} - G06F 3/0482: Interaction with lists of selectable items, e.g. menus - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract Disclosed are approaches for user interface control. In one embodiment, a method can include: indicating a highlight of a first object of a plurality of objects displayed on a display screen; accepting a signal deriving from a first gesture input of a user indicating a neutral position relative to the highlighted first object; accepting a signal deriving from a second gesture input of the user indicating a directional position at least a predetermined distance from the neutral position; and indicating, in response to the second gesture input, a movement of the highlight from the first object to a second object adjacent to the first object, and where the directional position of the second gesture input correlates to a position on the display screen of the second object relative to the first object. ## Claims ### Claim 1 A method of controlling a user interface, the method comprising: highlighting a first object on a display screen, wherein the first object is one of a plurality of objects displayed on the display screen; accepting, by a computing device, a signal deriving from a first gesture input of a user, wherein the first gesture input, provided by a body part of the user without making any direct contact with the display screen, indicates a neutral position of the body part relative to the highlighted first object on the display screen; providing on the display screen, in response to the first gesture input of the user, a graphical representation at a first position, the first position having a predetermined relationship with the position of the highlighted first object; accepting, by the computing device, a signal deriving from a second gesture input of the user, wherein the second gesture input, caused by movement of the body part through a gesture distance from the neutral position along a gesture direction from the neutral position, indicates a directional position; moving the graphical representation, in response to the second gesture input, to a second position on the display screen indicative of the directional position; detecting whether the movement of the graphical representation to the second position crosses a circular ring threshold, centered at the neutral position corresponding to the screen position of the highlighted first object and characterized by a predetermined radius, the circular ring threshold being independent of positioning of any of the plurality of objects other than the highlighted first object; and in response to detecting that the movement of the graphical representation to the second position does cross the circular ring threshold at a point on the display screen between the first object and a second object adjacent to the first object in the plurality of objects, highlighting the second object, wherein the directional position of the second gesture input correlates to a position on the display screen of the second object relative to the first object. ### Claim 2 (depends on 1) The method of claim 1, wherein the highlight comprises a dimension, the method further comprising: determining the movement of the highlight by using at least the directional position, the neutral position, and the dimension of the highlight. ### Claim 3 (depends on 1) The method of claim 1, further comprising: accepting, by the computing device a signal deriving from a third gesture input of the user, wherein the third gesture input indicates selection of the highlighted second object. ### Claim 4 (depends on 3) The method of claim 3, wherein the third gesture input comprises a hand of the user in a grasp state. ### Claim 5 (depends on 1) The method of claim 1, wherein the first gesture input comprises a hand of the user in a flat upright state. ### Claim 6 (depends on 5) The method of claim 5, wherein the second gesture input comprises the flat upright state hand of the user being moved according to the directional position. ### Claim 7 (depends on 1) The method of claim 1, further comprising: performing an initialization procedure prior to accepting the signal deriving from the first gesture input of the user. ### Claim 8 (depends on 1) The method of claim 1, wherein each of the plurality of objects represents media content that is playable on the display screen. ### Claim 9 (depends on 1) The method of claim 1, wherein: the movement of the highlight is performed at a first predetermined speed if the directional position lies between inner and outer concentric rings envisaged around the neutral position; and the movement of the highlight is performed at a second predetermined speed if the directional position exceeds the outer concentric ring, wherein the second predetermined speed is greater than the first predetermined speed. ### Claim 10 An apparatus configured to control a user interface, the apparatus comprising: one or more processors; and a computer-readable non-transitory storage medium having one or more instructions stored thereon for execution by the one or more processors, and when executed being operable to perform: highlighting a first object on a display screen, wherein the first object is one of a plurality of objects displayed on the display screen; accepting, by a computing device, a signal deriving from a first gesture input of a user, wherein the first gesture input, provided by a body part of the user without making any direct contact with the display screen, indicates a neutral position of the body part relative to the highlighted first object on the display screen; providing on the display screen, in response to the first gesture input of the user, a graphical representation at a first position coupled to the highlighted first object; accepting, by the computing device, a signal deriving from a second gesture input of the user, wherein the second gesture input, caused by movement of the body part through a gesture distance from the neutral position along a gesture direction from the neutral position, indicates a directional position; moving the graphical representation, in response to the second gesture input, to a second position on the display screen indicative of the directional position; detecting whether the movement of the graphical representation to the second position crosses a circular ring threshold, centered at the neutral position corresponding to the screen position of the highlighted first object, and characterized by a predetermined radius, the circular ring threshold being independent of positioning of any of the plurality of objects other than the highlighted first object; and in response to detecting that the movement of the graphical representation to the second position does cross the circular ring threshold at a point on the display screen between the first object and a second object adjacent to the first object in the plurality of objects, highlighting the second object, wherein the directional position of the second gesture input correlates to a position on the display screen of the second object relative to the first object. ### Claim 11 (depends on 10) The apparatus of claim 10, wherein the highlight comprises a dimension, the one or more instructions when executed being further operable to perform: determining the movement of the highlight by using at least the directional position, the neutral position, and the dimension of the highlight. ### Claim 12 (depends on 10) The apparatus of claim 10, the one or more instructions when executed being further operable to perform: accepting, by the computing device, a signal deriving from a third gesture input of the user, wherein the third gesture input indicates selection of the highlighted second object. ### Claim 13 (depends on 12) The apparatus of claim 12, wherein the third gesture input comprises a hand of the user in a grasp state. ### Claim 14 (depends on 10) The apparatus of claim 10, wherein the first gesture input comprises a hand of the user in a flat upright state, and wherein the second gesture input comprises the flat upright state hand of the user being moved according to the directional position. ### Claim 15 (depends on 10) The apparatus of claim 10, the one or more instructions when executed being further operable to perform: performing an initialization procedure prior to accepting the signal deriving from the first gesture input of the user. ### Claim 16 (depends on 10) The apparatus of claim 10, wherein each of the plurality of objects represents media content that is playable on the display screen. ### Claim 17 (depends on 1) The apparatus of claim 1, wherein: the movement of the highlight is performed at a first predetermined speed if the directional position lies between inner and outer concentric rings envisaged around the neutral position; and the movement of the highlight is performed at a second predetermined speed if the directional position exceeds the outer concentric ring, wherein the second predetermined speed is greater than the first predetermined speed. ### Claim 18 A system for controlling a user interface, the system comprising: means for indicating a highlight of a first object on a display screen, wherein the first object is one of a plurality of objects displayed on the display screen; means for accepting, by a computing device, a signal deriving from a first gesture input of a user, wherein the first gesture input, provided by a body part of the user without making any direct contact with the display screen, indicates a neutral position of the body part relative to the highlighted first object on the display screen; means for providing on the display screen, in response to the first gesture input of the user, a graphical representation at a first position coupled to the highlighted first object; means for accepting, by the computing device, a signal deriving from a second gesture input of the user, wherein the second gesture input, caused by movement of the body part through a gesture distance from the neutral position along a gesture direction from the neutral position, indicates a directional position; means for moving the graphical representation, in response to the second gesture input, to a second position on the display screen indicative of the directional; means for detecting whether the movement of the graphical representation to the second position crosses a circular ring threshold, centered at the neutral position corresponding to the screen position of the highlighted first object, and characterized by a predetermined radius, the circular ring threshold being independent of positioning of any of the plurality of objects other than the highlighted first object; and means for highlighting, in response to detecting that the movement of the graphical representation to the second position does cross the circular ring threshold at a point on the display screen between the first object and a second object adjacent to the first object in the plurality of objects, wherein the directional position of the second gesture input correlates to a position on the display screen of the second object relative to the first object.