# US 12,001,644 B2 · Method and apparatus for adding interactive objects to a virtual reality environment > Agents: this data is available as JSON from the patentgrep API and MCP server — see https://patentgrep.com/llms.txt - Number: US 12,001,644 B2 - Document ID: US12001644B2 - Status: Granted - Kind: B2 - Application number: 17/258,634 - Filed: 2019-07-12 - Published: 2021-09-16 - Granted: 2024-06-04 - Earliest claimed: 2018-07-24 - Assignees: Nokia Technologies Oy - Applicants: Nokia Technologies Oy - Inventors: Jussi Leppänen; Arto Lehtiniemi; Antti Eronen; Sujeet Shyamsundar Mate - Web page: https://patentgrep.com/patent/US12001644B2 ## CPC - G06F 3/04815: Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object - G06F 3/011: {Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00)} - G06F 3/013: {Eye tracking input arrangements (G06F3/015 takes precedence)} - G06F 3/0484: for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range - G06F 3/012: {Head tracking input arrangements} - G06F 3/016: {Input arrangements with force or tactile feedback as computer generated output to the user} - G06F 3/017: {Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883)} - G06F: ELECTRIC DIGITAL DATA PROCESSING (computer systems based on specific computational models G06N) ## Abstract An apparatus, method and computer program is described comprising adding an interactive object to an interactive object queue in response to a user indication to add the interactive object to the queue, providing a first control area corresponding to the interactive object in response to the interactive object being added to the interaction object queue, wherein the first control area is accessible to the user, determining whether the user is within the first control area, providing at least one feedback to the user in response to determining that the user is within the first control area, and receiving an interaction input from the user for interacting with the interactive object. ## Claims ### Claim 1 An apparatus comprising: at least one processor; and at least one non-transitory memory including computer program code, the at least one non-transitory memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform at least the following: add an interactive object to an interactive object queue in response to a user indication to add the interactive object to the interactive object queue, wherein the interactive object comprises a virtual object; provide a first control area, corresponding to the interactive object, in response to the interactive object being added to the interactive object queue; modify the first control area to be continuously accessible to the user in response to a movement of the user away from the interactive object, wherein at least one of a size or a shape of the first control area is modified relative to the user; determine whether the user is at least partially within the first control area while the interactive object is outside a field of vision of the user; and provide feedback to the user in response to determining that the user is at least partially within the first control area. ### Claim 2 (depends on 1) The apparatus as claimed in claim 1, wherein the adding the interactive object to the interactive object queue comprises the at least one non-transitory memory and the computer program code are configured to with the at least one processor, cause the apparatus to: determine a user gaze towards the interactive object, wherein the user gaze is the user indication. ### Claim 3 (depends on 2) The apparatus as claimed in claim 2, wherein the determining the user gaze comprises the at least one non-transitory memory and the computer program code are configured to with the at least one processor, cause the apparatus to determine at least one of whether: the interactive object is in the field of vision of the user for at least a first threshold time period, or a distance between the user and the interactive object is lower than a first threshold distance. ### Claim 4 (depends on 1) The apparatus as claimed in claim 1, wherein the at least one non-transitory memory and the computer program code are configured to, with the at least one processor, cause the apparatus to: receive an interaction input from the user, wherein the interaction input is configured to cause interaction with the interactive object, wherein the interaction input comprises an interaction input with respect to the first control area. ### Claim 5 (depends on 1) The apparatus as claimed in claim 1, wherein the first control area does not overlap with at least one existing control area corresponding to at least one existing interactive object in the interactive object queue. ### Claim 6 (depends on 1) The apparatus as claimed in claim 1, wherein the at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus to remove at least one interactive object from the interactive object queue. ### Claim 7 (depends on 6) The apparatus as claimed in claim 6, wherein the at least one interactive object is removed when the user does not interact with the at least one interactive object for a second threshold time period. ### Claim 8 (depends on 1) The apparatus as claimed in claim 1, wherein the at least one non-transitory memory and the computer program code are further configured to, with the at least one processor, cause the apparatus to provide a visual representation of the interactive object to a view of the user, wherein the visual representation is configured to indicate that the interactive object has been added to the interactive object queue. ### Claim 9 (depends on 8) The apparatus as claimed in claim 8, wherein a size of the visual representation of the interactive object is increased when adding the interactive object to the interactive object queue. ### Claim 10 (depends on 1) The apparatus as claimed in claim 1, wherein the at least one non-transitory memory and the computer program code are further configured to, with the at least one processor, cause the apparatus to provide a visual representation of the first control area to the user. ### Claim 11 (depends on 1) The apparatus as claimed in claim 1, wherein the at least one non-transitory memory and the computer program code are further configured to, with the at least one processor, cause the apparatus to notify the user when the interactive object is added to the interactive object queue. ### Claim 12 (depends on 1) The apparatus as claimed in claim 1, wherein the at least one non-transitory memory and the computer program code are further configured to, with the at least one processor, cause the apparatus to: receive a rejection input from the user within a third threshold time period after adding at least one interactive object to the interactive object queue; and remove the at least one interactive object from the interactive object queue on receipt of the rejection input. ### Claim 13 A method comprising: adding an interactive object to an interactive object queue in response to a user indication to add the interactive object to the interactive object queue, wherein the interactive object comprises a virtual object; providing a first control area, corresponding to the interactive object, in response to the interactive object being added to the interactive object queue; modifying the first control area to be continuously accessible to the user in response to a movement of the user away from the interactive object, wherein at least one of a size or a shape of the first control area is modified relative to the user; determining whether the user is at least partially within the first control area while the interactive object is outside a field of vision of the user; and providing at least one feedback to the user in response to determining that the user is at least partially within the first control area. ### Claim 14 (depends on 13) The method as claimed in claim 13, wherein the adding the interactive object to the interactive object queue comprises determining a user gaze towards the interactive object, wherein the user gaze is the user indication. ### Claim 15 (depends on 14) The method as claimed in claim 14, wherein the determining the user gaze comprises determining at least one of whether: the interactive object is in the field of vision of the user for at least a first threshold time period, or a distance between the user and the interactive object is lower than a first threshold distance. ### Claim 16 (depends on 13) The method as claimed in claim 13, further comprising: receiving an interaction input from the user, wherein the interaction input is configured to cause interaction with the interactive object, wherein the interaction input comprises an interaction input with respect to the first control area. ### Claim 17 (depends on 13) The method as claimed in claim 13, wherein the providing the first control area comprises providing the first control area such that the first control area does not overlap with at least one existing control area corresponding to at least one existing interactive object in the interactive object queue. ### Claim 18 (depends on 13) The method as claimed in claim 13, further comprising removing at least one interactive object from the interactive object queue. ### Claim 19 (depends on 18) The method as claimed in claim 18, wherein the at least one interactive object is removed when the user does not interact with the at least one interactive object for a second threshold time period. ### Claim 20 A non-transitory computer readable medium comprising program instructions stored thereon for performing at least the following: adding an interactive object to an interactive object queue in response to a user indication to add the interactive object to the interactive object queue, wherein the interactive object comprises a virtual object; providing a first control area, corresponding to the interactive object, in response to the interactive object being added to the interactive object queue; modifying the first control area to be continuously accessible to the user in response to a movement of the user away from the interactive object, wherein at least one of a size or a shape of the first control area is modified relative to the user; determining whether the user is at least partially within the first control area while the interactive object is outside a field of vision of the user; and providing at least one feedback to the user in response to determining that the user is at least partially within the first control area.