# US 10,120,437 B2 · Methods and systems for associating input schemes with physical world objects > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 10,120,437 B2 - Document ID: US10120437B2 - Status: Granted - Kind: B2 - Application number: 15/011,119 - Filed: 2016-01-29 - Published: 2017-08-03 - Granted: 2018-11-06 - Earliest claimed: 2016-01-29 - Assignees: Rovi Guides, Inc. - Applicants: Rovi Guides, Inc. - Inventors: Edison Lin - Web page: https://patentgrep.com/patent/US10120437B2 ## CPC - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F 3/167: {Audio in a user interface, e.g. using voice commands for navigating, audio feedback} - H04N 21/41407: {embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop (constructional details of equipment or arrangements specially adapted for portable computer application G06F1/1626; arrangements specially adapted for mobile receivers in broadcast systems H04H20/57)} - H04N 21/4223: Cameras (H04N23/00 takes precedence) - H04N 21/4532: {involving end-user characteristics, e.g. viewer profile, preferences (monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18)} - H04N 21/47202: {for requesting content on demand, e.g. video on demand} - H04N 21/4755: {for defining user preferences, e.g. favourite actors or genre (retrieval personalisation and generation of user profiles for the retrieval of video data G06F16/739; user profiles in network data switching protocols H04L67/306)} - H04N 21/4821: {using a grid, e.g. sorted out by channel and broadcast time} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) - H04N: PICTORIAL COMMUNICATION, e.g. TELEVISION ## Abstract A media guidance application that allows users to associate input schemes with physical objects in an augmented reality environment is disclosed. Specifically, the media guidance application may recognize physical objects in an augmented reality environment and allow users to identify input schemes to associate with the physical objects. Such input schemes may define ways in which the users may control presentation of media content by interacting with the physical objects. ## Claims ### Claim 1 A method for controlling presentation of media content with physical world objects, the method comprising: storing a first digital description of a first physical object in an augmented reality environment of a user; receiving a user input from the user assigning an input scheme for controlling a presentation of media content to the first physical object by: receiving a voice input from the user identifying an attribute of the presentation of the media content to be controlled; detecting a user gesture occurring simultaneously with the receiving of the voice input, identifying an interaction with the first physical object that controls the attribute; and associating the detected user gesture with the attribute identified by the voice input from the user based on the user gesture occurring simultaneously with the receiving of the voice input; and after receiving the user input assigning the input scheme for controlling presentation of media content to the first physical object: detecting the first physical object in the augmented reality environment by comparing digital descriptions of physical objects in the augmented reality environment to the first digital description of the first physical object; and in response to detecting the first physical object in the augmented reality environment, controlling presentation of media content using the input scheme. ### Claim 2 (depends on 1) The method of claim 1, wherein the first digital description of the first physical object is created using image segmentation and edge detection on an image of the first physical object. ### Claim 3 (depends on 1) The method of claim 1, wherein the first digital description of the first physical object is an alphanumeric value describing the shape and color of the first physical object. ### Claim 4 (depends on 1) The method of claim 1, wherein the user initially identifies the first physical object by performing a hand gesture over the first physical object. ### Claim 5 (depends on 1) The method of claim 1, wherein the input scheme for controlling presentation of media content is one of a plurality of predefined input schemes, each associated with a physical object. ### Claim 6 (depends on 1) The method of claim 1, wherein the user input assigning an input scheme for controlling presentation of media content to the first physical object includes navigating to the input scheme in a list of predefined input schemes. ### Claim 7 (depends on 1) The method of claim 1, wherein storing a first digital description of a first physical object in an augmented reality environment comprises: creating an object identifier associated with the first physical object; and storing the object identifier and the first digital description in a database listing object identifiers, each corresponding to a digital description. ### Claim 8 (depends on 7) The method of claim 7, further comprising storing a media asset identifier associated with the media asset and the object identifier in a database listing object identifiers, each corresponding to a media asset. ### Claim 9 (depends on 1) The method of claim 1 wherein controlling the presentation of the media content using the input scheme comprises: detecting an additional user gesture that matches the user gesture identifying the interaction with the first physical object that controls the attribute; and adjusting the attribute of the presentation of the media content based on detecting the additional user gesture. ### Claim 10 (depends on 9) The method of claim 9, wherein the attribute of the presentation of the media content comprises at least one of: volume of the presentation of the media content, playback speed of the presentation of the media content, and pause/play functionality of the presentation of the media content. ### Claim 11 A system for controlling presentation of media content with physical world objects, the system comprising: storage circuitry configured to store a first digital description of a first physical object in an augmented reality environment of a user; and control circuitry configured to: receive a user input from the user assigning an input scheme for controlling a presentation of media content to the first physical object by: receiving a voice input from the user identifying an attribute of the presentation of the media content to be controlled; detecting a user gesture occurring simultaneously with the receiving of the voice input, identifying an interaction with the first physical object that controls the attribute; and associating the detected user gesture with the attribute identified by the voice input from the user based on the user gesture occurring simultaneously with the receiving of the voice input; and after receiving the user input assigning the input scheme for controlling presentation of media content to the first physical object: detect the first physical object in the augmented reality environment by comparing digital descriptions of physical objects in the augmented reality environment to the first digital description of the first physical object; and in response to detecting the first physical object in the augmented reality environment, control presentation of media content using the input scheme. ### Claim 12 (depends on 11) The system of claim 11, wherein the first digital description of the first physical object is created using image segmentation and edge detection on an image of the first physical object. ### Claim 13 (depends on 11) The system of claim 11, wherein the first digital description of the first physical object is an alphanumeric value describing the shape and color of the first physical object. ### Claim 14 (depends on 11) The system of claim 11, wherein the user initially identifies the first physical object by performing a hand gesture over the first physical object. ### Claim 15 (depends on 11) The system of claim 11, wherein the input scheme for controlling presentation of media content is one of a plurality of predefined input schemes, each associated with a physical object. ### Claim 16 (depends on 11) The system of claim 11, wherein the user input assigning an input scheme for controlling presentation of media content to the first physical object includes navigating to the input scheme in a list of predefined input schemes. ### Claim 17 (depends on 11) The system of claim 11, wherein control circuitry is further configured, when storing a first digital description of a first physical object in an augmented reality environment, to create an object identifier associated with the first physical object; and storage circuitry further configured to store the object identifier and the first digital description in a database listing object identifiers, each corresponding to a digital description. ### Claim 18 (depends on 17) The system of claim 17, further comprising storing a media asset identifier associated with the media asset and the object identifier in a database listing object identifiers, each corresponding to a media asset. ### Claim 19 (depends on 11) The system of claim 11 wherein the control circuitry is configured to control the presentation of the media content using the input scheme by: detecting an additional user gesture that matches the user gesture identifying the interaction with the first physical object that controls the attribute; and adjusting the attribute of the presentation of the media content based on detecting the additional user gesture. ### Claim 20 (depends on 19) The system of claim 19, wherein the attribute of the presentation of the media content comprises at least one of: volume of the presentation of the media content, playback speed of the presentation of the media content, and pause/play functionality of the presentation of the media content.